-- Logs begin at Thu 2022-09-01 06:35:53 UTC, end at Thu 2022-09-01 08:03:59 UTC. -- Sep 01 06:59:54.360237 dv-cinder-733062-35-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 06:59:54.371543 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 06:59:54 2022] *** Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: nodename: dv-cinder-733062-35-iscsi Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: machine: x86_64 Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: clock source: unix Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: pcre jit disabled Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: detected number of CPU cores: 4 Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: current working directory: / Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: detected binary path: /usr/bin/uwsgi-core Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: your processes number limit is 47680 Sep 01 06:59:54.373797 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: your memory page size is 4096 bytes Sep 01 06:59:54.374448 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: detected max file descriptor number: 2048 Sep 01 06:59:54.374448 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: lock engine: pthread robust mutexes Sep 01 06:59:54.374448 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: thunder lock: enabled Sep 01 06:59:54.374448 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 06:59:54.374448 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 06:59:54.412935 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: Python main interpreter initialized at 0x56011f2da9c0 Sep 01 06:59:54.412935 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: python threads support enabled Sep 01 06:59:54.412935 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: your server socket listen backlog is limited to 100 connections Sep 01 06:59:54.412935 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: your mercy for graceful operations on workers is 90 seconds Sep 01 06:59:54.413341 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 06:59:54.413422 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: *** Operational MODE: preforking *** Sep 01 06:59:54.413644 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: *** uWSGI is running in multiple interpreter mode *** Sep 01 06:59:54.413644 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: spawned uWSGI master process (pid: 81597) Sep 01 06:59:54.414350 dv-cinder-733062-35-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 06:59:54.417143 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: spawned uWSGI worker 1 (pid: 81610, cores: 1) Sep 01 06:59:54.417143 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: spawned uWSGI worker 2 (pid: 81611, cores: 1) Sep 01 06:59:54.417143 dv-cinder-733062-35-iscsi devstack@keystone.service[81597]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 06:59:58.728052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 06:59:58.728052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 01 06:59:58.728842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 06:59:58.728842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Sep 01 06:59:58.729287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 06:59:58.729287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 01 06:59:58.729397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 06:59:58.729397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Sep 01 06:59:58.778179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 06:59:58.778375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 06:59:58.778568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] command line args: [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 06:59:58.778780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 06:59:58.778867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 06:59:58.778977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 06:59:58.779067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ================================================================================ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 06:59:58.779173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] command line args: [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 06:59:58.779257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.779445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 06:59:58.779528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_dir = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.779630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ================================================================================ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 06:59:58.779717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.779816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_source = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.779893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.779999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] debug = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.780129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_dir = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.780216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.781824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.781824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] config_source = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] debug = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.782849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_config_append = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_dir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.783967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_config_append = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_dir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.784940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.785940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.786953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] publish_errors = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.787950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] publish_errors = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] standard_threads = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.788777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] standard_threads = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_journal = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_json = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.789950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_stderr = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_syslog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_journal = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_json = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.790455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_stderr = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] use_syslog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.790662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 06:59:58.790742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.790813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.790888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.790963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.791975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.792916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.793922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.794884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.795991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.796970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.797948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.798989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.799918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.800916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.801953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.802964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.803913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.804033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.804148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.804261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.804383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.805977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.806904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.807947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.808987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.809905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.810905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.811822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.812959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.813901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.814949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.external = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.815957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.password = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.token = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.external = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.816998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.password = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.token = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.817983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.818924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.819989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.820936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.821896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.822955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.823896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.824946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.825988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.826917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.827986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.828916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.829986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.830934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.831977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.832913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.833909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.834973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.835910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.836952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.837909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.838953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.839893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.840940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.841924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.842975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.843903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.844961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.845906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.846941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.847984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.848986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.849970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.850899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.851013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.851136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.851253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.851873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.851988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.852902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.853900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.854962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.855897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856126 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.856953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.857938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.858978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.859919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.860974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.861960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.862910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.863965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.864895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 06:59:58.865897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 06:59:58.894345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.894520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.894605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.894751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.894833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.894975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.903132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.903589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.932042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.932263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.932710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.932942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.985576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:58.986100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.038447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.038675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.039294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.039532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.194987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.195673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.195773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.195895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.196111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.218593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.218792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.218877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.219965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.220099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.230694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.231331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.244890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.245023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.245105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.245265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.245347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.245480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 06:59:59.262164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.262374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 06:59:59.262616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.262699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.262828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 06:59:59.262924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 06:59:59.263162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.263242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.263360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 06:59:59.263440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.263637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.263719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.263920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.264152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.264420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.264683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.274151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.274432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.274705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.274786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.274987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.275080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.275262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.275548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.282042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.282583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.284955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.285272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.285517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.285611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.285806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.285887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.286986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.287839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.288122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.288398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.312160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.312790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.313671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.313961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.314315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.314602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.318539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.318826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.319112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.319605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.319900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.320211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.325688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.325978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.326668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.326959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.329998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.330292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.330577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.330875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.331109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.331202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.331404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.331691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.331977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.332280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.332568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.332840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.333062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.333158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.333372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.333661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.349907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.350189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.350964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.351249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.354660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.355711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.357808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.358108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.359012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.359313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.362499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.363585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.365659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.366780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.369051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.369376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.369669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.369958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.370252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.370332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.370628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.370709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.370919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.371209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.371512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.371797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.386534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.386824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.387095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.387369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.387779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.388075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.388350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.388623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.394194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.395615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.395848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.397175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.397427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.397706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.399275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.399556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.400915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.401226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.401516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.401807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.402099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.402266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.402556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.402954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.403245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.403542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.413556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.413902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.414678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.414962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.415246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.415443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.415525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.415739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.415819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.416121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.416411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.416702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.416988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.417550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.431436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.433424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.434537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.436383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.437470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.438925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.439335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.440413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.440700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.440806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.440991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.442449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.442741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.443044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.447653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.449146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.449652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.451145 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.452112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.452407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.452694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.452975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.454221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.454521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.454812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.455091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.461498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.461791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.462075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.463542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.463837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.464125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.469055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.469379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.469662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.469938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.470210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.470481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.470771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 06:59:59.471326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.471963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.472963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.473276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.473557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.473831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.474098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.474363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.474640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 06:59:59.507431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56011f2da9c0 pid: 81611 (default app) Sep 01 06:59:59.510172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56011f2da9c0 pid: 81610 (default app) Sep 01 06:59:59.511279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067975a-b797-4020-b8bd-facbe6785c93 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 06:59:59.511567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067975a-b797-4020-b8bd-facbe6785c93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 06:59:59.511845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067975a-b797-4020-b8bd-facbe6785c93 None None] PATH_INFO: `/v3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 06:59:59.516018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu Sep 1 06:59:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:06.005204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:06.005705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:06.005811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:06.010621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:06.015873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:06.016295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:06.016433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:06.086358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: INFO dbcounter [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] Registered counter for database keystone Sep 01 07:00:06.105083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG oslo_db.sqlalchemy.engines [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81611) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 07:00:06.111232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writer thread running {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 07:00:06.292951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81611) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 07:00:06.293370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81611) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 07:00:06.350490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:06.352805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:06.480349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 07:00:06.934621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:07.003408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: INFO dbcounter [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] Registered counter for database keystone Sep 01 07:00:07.025120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG oslo_db.sqlalchemy.engines [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81610) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 07:00:07.031178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writer thread running {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 07:00:07.148898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9af849-5d7c-4c56-bde6-2daa127562e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:07.153101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:07.153737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:07.154018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:07.154285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:07.162165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:07.213935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING keystone.common.rbac_enforcer.enforcer [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 07:00:07.214274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4d540-d04f-4e4a-b744-149b610312fb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:07.223566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Sep 1 07:00:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 07:00:07.229526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:07.243852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06b84d-9e70-4a0c-8730-4c844a1d56e6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:07.248435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:07.249578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:07.249860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:07.250130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] PATH_INFO: `/v3/projects/b911f14b05394fe89de7092f0075d554` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:07.251697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] RBAC: Authorizing `identity:get_project(project_id=b911f14b05394fe89de7092f0075d554)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:07.304560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING keystone.common.rbac_enforcer.enforcer [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 07:00:07.304855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d67814-0fc6-4706-95a7-36bb314bde25 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:07.309382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu Sep 1 07:00:07 2022] GET /identity/v3/projects/b911f14b05394fe89de7092f0075d554 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:09.164705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c42e0-9272-4da9-85f1-c914dbebb08b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:09.165251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c42e0-9272-4da9-85f1-c914dbebb08b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:09.165338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c42e0-9272-4da9-85f1-c914dbebb08b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:09.169379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:09.174938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:09.175310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:09.175490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:09.196273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:09.197826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:09.205601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 07:00:09.470245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c42e0-9272-4da9-85f1-c914dbebb08b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:09.482991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c42e0-9272-4da9-85f1-c914dbebb08b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:09.487198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:09.487851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:09.488130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:09.488402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:09.496639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:09.498430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8c2a7-78f2-41a5-bfc9-b42d424ccf9b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:09.507032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Sep 1 07:00:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 07:00:09.513130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:09.522433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6eeba5-eff7-4605-94ee-5ae7328701fe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:09.527465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:09.528090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:09.528370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:09.528646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:09.529967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:09.531792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965df6bd-eded-4f7e-a26b-7df96d07e287 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:09.533336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu Sep 1 07:00:09 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:11.475244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb1e6-070b-4b2c-ab5b-4ce99886a26b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:11.475812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb1e6-070b-4b2c-ab5b-4ce99886a26b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:11.475812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeb1e6-070b-4b2c-ab5b-4ce99886a26b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:11.477675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:11.482876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:11.483327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:11.483428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:11.501967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:11.503124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:11.529743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affacb-181b-486d-a221-1b4a7542910d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:11.530027 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affacb-181b-486d-a221-1b4a7542910d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:11.530303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affacb-181b-486d-a221-1b4a7542910d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:11.531617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:11.536629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:11.536912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:11.537217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:11.579266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81610) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 07:00:11.579653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81610) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 07:00:11.582876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:11.631731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:11.633423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:11.732628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:11.852938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:11.877075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:11.954657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e39dc-5763-44f2-beea-53a84be51d8f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:11.987583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:11.988151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:11.988431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:11.988702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:11.998201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:11.999948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:12.009048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:12.010217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:12.011248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:12.015951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.097177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.100141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a53bfa-753f-4166-af49-77ad94bc78ff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.103180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.115603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d59638-1748-42fa-8ccf-a89bf278535b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.139460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.140317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.140605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.140882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.149246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.149843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.150120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.150387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.151047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:12.151346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.152742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.164936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:12.170373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.207923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.209482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.268635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f0056-d097-4bda-b815-e04bf9fb90be None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.302005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.302638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.302916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.303188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.304352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.305844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.307420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 9/16] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:12.307667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 8/17] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:12.312780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.313301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.316022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.328108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.410771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db45b5-7987-4223-b314-df739436ac54 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.411921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0644f0-72c3-4ba1-a505-8c409b2e1ca9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.443656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.444494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.444772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.445573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.446677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.447301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.447585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.447858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.448973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.450449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.452008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:12.457759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.466362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.467837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:00:12.485661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:12.487281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:00:12.488385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:00:12.488385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:00:12.488385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:12.488385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:12.492714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:12.542099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.548991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:12.562462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6759f-163c-48ff-9710-cb83a60143cb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.594042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514d8b4-f4e7-468c-901b-1d76c96ebfc7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:12.595780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.596882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.596882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.597078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.597713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.domain.id=default, user.id=82706f28e6f34da4b5745e5a95638706) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:12.598023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.599468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b06756-b726-4d39-94dc-693bc96109bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.606436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/role_assignments?user.id=82706f28e6f34da4b5745e5a95638706&scope.domain.id=default&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:12.630831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:12.631455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:12.631733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:12.632001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:12.633292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:12.633596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:12.635210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d937323-5532-44d9-bd69-06bf26574709 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:12.655430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:14.621110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a47574-3fe6-4679-aa25-e1dc22f7aee7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:14.622180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a47574-3fe6-4679-aa25-e1dc22f7aee7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:14.622672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a47574-3fe6-4679-aa25-e1dc22f7aee7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:14.624208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:14.629691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172d7c-d65f-4f19-8beb-31207fee224c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:14.630505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172d7c-d65f-4f19-8beb-31207fee224c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:14.630998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5172d7c-d65f-4f19-8beb-31207fee224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:14.657350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e5172d7c-d65f-4f19-8beb-31207fee224c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:14.658895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e5172d7c-d65f-4f19-8beb-31207fee224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:14.755378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:15.171375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39198-106a-4974-86f5-ae4dae30e969 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.171667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39198-106a-4974-86f5-ae4dae30e969 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.171927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe39198-106a-4974-86f5-ae4dae30e969 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.173454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:15.178302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf0a68-00e1-4b7a-9aed-bdecbdeaf7a3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.178591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf0a68-00e1-4b7a-9aed-bdecbdeaf7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.178870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf0a68-00e1-4b7a-9aed-bdecbdeaf7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.201643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bbdf0a68-00e1-4b7a-9aed-bdecbdeaf7a3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:15.202823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bbdf0a68-00e1-4b7a-9aed-bdecbdeaf7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:15.255235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:15.291616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 12/25] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:15.304148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe39198-106a-4974-86f5-ae4dae30e969 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:15.368713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8fbd-5e53-495f-ad0c-7d84fcf944f8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.369011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8fbd-5e53-495f-ad0c-7d84fcf944f8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.369293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf8fbd-5e53-495f-ad0c-7d84fcf944f8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.370768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 13/26] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:15.375809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407f05-38a6-4add-9210-40f500d3eb1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.376352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407f05-38a6-4add-9210-40f500d3eb1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.376829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0407f05-38a6-4add-9210-40f500d3eb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.406551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f0407f05-38a6-4add-9210-40f500d3eb1b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:15.407736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0407f05-38a6-4add-9210-40f500d3eb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:15.482194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe39198-106a-4974-86f5-ae4dae30e969 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:15.517064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 14/27] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:15.518348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cfa3-5b09-4711-b1b3-befe70409cc8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.518634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cfa3-5b09-4711-b1b3-befe70409cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.518903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9cfa3-5b09-4711-b1b3-befe70409cc8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.520313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 15/28] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:15.531892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.532170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.532444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.555927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:15.557720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:15.564271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:15.564950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.565260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.565527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.584903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:15.594298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:15.595140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:15.595909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 14/29] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:15.602615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:15.610168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:15.699079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 16/30] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:15.701622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:15.781464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:15.809152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dab10-5444-4ac5-b8d9-a29d1a546bf7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:15.879984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e25d-77f1-41b0-9677-fa34531cdf91 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:15.882429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:15.883574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.883851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.884121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.884796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:15.885100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:15.886519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:15.907312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 15/31] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:15.908517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:15.939810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:15.940687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:15.941108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:15.941609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:15.963043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:15.964864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:00:15.997684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:15.998599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:00:15.999403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:00:15.999403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:00:15.999403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:16.006657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 17/32] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:16.007727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.041529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.104197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0724-0b82-4485-8ec5-d866297d1a0d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.148932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.149953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.150455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.150926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.151793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.153434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.163349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.178193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0cc2-799a-42aa-aacc-f7d9303de9df None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.216692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.217639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.218062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.218478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.220222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.221946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.223684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 18/33] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:16 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:16.224669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.318192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 16/34] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 07:00:15 2022] POST /identity/v3/roles => generated 225 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:16.319613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.345384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906c55a-7757-470e-b6c0-9329711c58e4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.388916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.389891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.390379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.390815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.391623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:16.392073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.393753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.409596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 19/35] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:16.411400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.439966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.462454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002151-e799-4470-aae2-5f5496c20658 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.496484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.509582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.510783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.511143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.511440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.512121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.513734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f20936-fc77-4d67-b23a-ae8542a5e4a4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.533527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 17/36] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 07:00:16 2022] POST /identity/v3/roles => generated 219 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:16.534849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87a5c-13b3-400b-8668-cec30ca49a9b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.535140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87a5c-13b3-400b-8668-cec30ca49a9b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.535419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe87a5c-13b3-400b-8668-cec30ca49a9b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.536639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 18/37] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:16.542342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe87a5c-13b3-400b-8668-cec30ca49a9b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.543482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc30333-9acc-4b59-a68b-0efe55304ec3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.582336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.582907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.583184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.583453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.584701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.586785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2236-f6d1-439f-9185-f88ee48bb3a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 20/38] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b382ffe-7679-4950-8d67-2d6257ceb893 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b382ffe-7679-4950-8d67-2d6257ceb893 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b382ffe-7679-4950-8d67-2d6257ceb893 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 21/39] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd54e4-e90b-4330-933d-3b1017efaa62 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.593443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd54e4-e90b-4330-933d-3b1017efaa62 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.593966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd54e4-e90b-4330-933d-3b1017efaa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.613568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-77bd54e4-e90b-4330-933d-3b1017efaa62 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:16.614804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-77bd54e4-e90b-4330-933d-3b1017efaa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:16.663371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe87a5c-13b3-400b-8668-cec30ca49a9b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.693955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 22/40] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:16.695068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.695361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.695638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.705110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.705765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.706050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.706327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.706982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.708440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.729671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:16.730871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:16.743059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.749143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 19/41] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:00:16 2022] POST /identity/v3/projects => generated 301 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:16.749143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:16.807333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:16.838555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 23/42] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:16.886763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76505f-7084-4c5a-bba0-f593adb82e7a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:16.925303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:16.925977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:16.926258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:16.926534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] PATH_INFO: `/v3/domains/default/users/82706f28e6f34da4b5745e5a95638706/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:16.928482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=82706f28e6f34da4b5745e5a95638706, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:16.930020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:16.948303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 20/43] 10.222.0.46 () {58 vars in 1361 bytes} [Thu Sep 1 07:00:16 2022] PUT /identity/v3/domains/default/users/82706f28e6f34da4b5745e5a95638706/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:16.993611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:17.106136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99b8c4-3263-4d88-a1d3-6985c91290d4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:17.113910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:17.146785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:17.147453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:17.147911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:17.148040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:17.149175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:17.150635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:17.152381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 24/44] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:17.157619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:17.176359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:17.227540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:17.232126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53062c3f-c373-4f41-926b-e33b9c59655f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:17.267967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:17.268568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:17.268846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:17.269137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:17.269821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:17.271219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50116d-251c-472b-abaa-fbc7a38b566c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:17.276868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801c4fa-2d3c-4bd9-a654-a0d0c54d49d7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:17.286836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 21/45] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 07:00:17 2022] POST /identity/v3/roles => generated 223 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:00:17.318458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:17.319165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:17.319447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:17.319718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:17.320999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:17.322521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4840269-f41d-4dfd-bb04-428310162cad None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:17.469089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 25/46] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:00:17 2022] POST /identity/v3/projects => generated 312 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:00:18.860182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825435e9-7fc3-4e7e-b10b-e501010154bd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:18.860679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825435e9-7fc3-4e7e-b10b-e501010154bd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:18.860785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825435e9-7fc3-4e7e-b10b-e501010154bd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:18.862121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 22/47] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:18.868282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89f85-8a10-4a91-a020-fcbe02c6f584 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:18.868708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89f85-8a10-4a91-a020-fcbe02c6f584 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:18.868838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89f85-8a10-4a91-a020-fcbe02c6f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:18.888019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b8f89f85-8a10-4a91-a020-fcbe02c6f584 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:18.889560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b8f89f85-8a10-4a91-a020-fcbe02c6f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:18.947630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:18.956777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 26/48] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:19.221216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825435e9-7fc3-4e7e-b10b-e501010154bd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.321953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825435e9-7fc3-4e7e-b10b-e501010154bd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:19.354156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:19.354786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.355069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.355346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.365208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:19.383974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82df6a0-4c75-4a6e-9396-8cec2d573616 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.384257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82df6a0-4c75-4a6e-9396-8cec2d573616 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.384534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82df6a0-4c75-4a6e-9396-8cec2d573616 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.386017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 27/49] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:19.391752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.392105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.392385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:19.479940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:19.481036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:19.482077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 23/50] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:19.487217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.507316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:19.516113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:19.517354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:19.582953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 28/51] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:19.589265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86a1b-5172-4ba2-805c-c3f520880f52 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:19.622929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:19.623538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.623822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.624095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.624759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:19.625062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:19.626467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:19.635156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 24/52] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:19.640684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.736475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb7e203-7e1c-4740-93f2-11f5277a595b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:19.759677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:19.771636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:19.772273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.772559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.772838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.774024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:19.775606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:19.777384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 29/53] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:19.782860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.852651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.862819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:19.883707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15f51f-1b6a-477c-85be-98bd3789abae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:19.919461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:19.920104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.920373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.920649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.921781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:19.923229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:19.925182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 25/54] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:19.931457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:19.952972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd454ac1-968d-4d6a-bf6c-74c304408b0b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:19.986127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:19.986800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:19.987031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:19.987306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:19.988403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:19.989867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:19.991422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 30/55] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:19.997648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:20.028091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9436-bbe0-4f5d-8c06-776220bca692 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:20.053652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:20.062741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:20.063342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:20.063622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:20.063894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:20.064609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.domain.id=default, user.id=82706f28e6f34da4b5745e5a95638706) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:20.064912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:20.066398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252201-d7ac-46ff-80db-b030e617f43b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:20.071982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 26/56] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Sep 1 07:00:19 2022] GET /identity/v3/role_assignments?user.id=82706f28e6f34da4b5745e5a95638706&scope.domain.id=default&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:20.097534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267f38-f18f-49b2-b85b-7f3af5ce7109 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:20.136307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:20.136938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:20.137256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:20.137537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:20.138206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:20.139697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962e40c-34ef-4629-8c5e-c060c929ad8d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:20.169603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 31/57] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Sep 1 07:00:19 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:00:22.076792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57fe14-249a-44d1-b2b9-da2eb6f651d0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:22.077355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57fe14-249a-44d1-b2b9-da2eb6f651d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:22.077355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57fe14-249a-44d1-b2b9-da2eb6f651d0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:22.078730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 27/58] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:22.083798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f709-21ee-467d-a202-ba1349665684 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:22.084194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f709-21ee-467d-a202-ba1349665684 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:22.084351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724f709-21ee-467d-a202-ba1349665684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:22.102413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1724f709-21ee-467d-a202-ba1349665684 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:22.103592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1724f709-21ee-467d-a202-ba1349665684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:22.166940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 32/59] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:22.437508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57fe14-249a-44d1-b2b9-da2eb6f651d0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:22.453146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:22.539268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57fe14-249a-44d1-b2b9-da2eb6f651d0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:22.571041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:22.571658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:22.571940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:22.572215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:22.573349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:22.574808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186b4d4-f4a7-40bd-93cd-3c3e2ea2bf28 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:22.576355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 28/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:22.581683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724f709-21ee-467d-a202-ba1349665684 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:22.623337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:22.678104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724f709-21ee-467d-a202-ba1349665684 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:22.709640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:22.710174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:22.710970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:22.711243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:22.711879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:22.713322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4dcca6-510c-4056-a730-cf31e993808a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:22.739047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 33/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:22 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:25.541408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d3254-1d09-4890-b2ed-54eb8c684550 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:25.541937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d3254-1d09-4890-b2ed-54eb8c684550 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:25.541937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d3254-1d09-4890-b2ed-54eb8c684550 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:25.543295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 29/62] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:25.549092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa90e7-4298-48b4-9a63-079e9a23660c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:25.549957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa90e7-4298-48b4-9a63-079e9a23660c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:25.550438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa90e7-4298-48b4-9a63-079e9a23660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:25.578085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0bfa90e7-4298-48b4-9a63-079e9a23660c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:25.579589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0bfa90e7-4298-48b4-9a63-079e9a23660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:25.674435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 34/63] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:25.899839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c04c-9ed3-4a4a-8b9e-5e26d2694e4a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:25.900447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c04c-9ed3-4a4a-8b9e-5e26d2694e4a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:25.900925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c04c-9ed3-4a4a-8b9e-5e26d2694e4a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:25.902507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 30/64] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:25.907689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c74a-ae0b-405e-a4fc-98c88428aaae None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:25.908219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c74a-ae0b-405e-a4fc-98c88428aaae None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:25.908704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c74a-ae0b-405e-a4fc-98c88428aaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:25.930337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d274c74a-ae0b-405e-a4fc-98c88428aaae None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:25.931850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d274c74a-ae0b-405e-a4fc-98c88428aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:25.963938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.023954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 35/65] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:26.091560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c04c-9ed3-4a4a-8b9e-5e26d2694e4a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.189370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227ed9-1f53-487f-b0a8-ccd824d59a18 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.189970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227ed9-1f53-487f-b0a8-ccd824d59a18 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.190491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227ed9-1f53-487f-b0a8-ccd824d59a18 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.192067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 36/66] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:26.202103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c080b-a7dd-4728-8785-50b72a260a70 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.203095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c080b-a7dd-4728-8785-50b72a260a70 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.203583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c080b-a7dd-4728-8785-50b72a260a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.206768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.227380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c02c080b-a7dd-4728-8785-50b72a260a70 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:26.229063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c02c080b-a7dd-4728-8785-50b72a260a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:26.253912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c04c-9ed3-4a4a-8b9e-5e26d2694e4a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.318172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.318817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.319097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.319372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.334606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:26.340698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:26.341607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:26.342353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 31/67] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:26.342977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.347650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 37/68] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:26.349029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7468-1b66-4ccc-9b9f-9bc7e91b3fb0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.349356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7468-1b66-4ccc-9b9f-9bc7e91b3fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.349641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7468-1b66-4ccc-9b9f-9bc7e91b3fb0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.350887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 38/69] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:26.352703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8eda-6c19-4a12-802d-d24f8c09770b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.352992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8eda-6c19-4a12-802d-d24f8c09770b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.353299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c8eda-6c19-4a12-802d-d24f8c09770b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.354559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 39/70] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:26.355479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8eda-6c19-4a12-802d-d24f8c09770b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.452833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.455504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b14fd8-a821-4bea-96f0-8423f107682d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.469583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c8eda-6c19-4a12-802d-d24f8c09770b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.492222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.493287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.493574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.493853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.501883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.505667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.506023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:26.506277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.506565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.506841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.508029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:26.508334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:26.509762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fa9d6-4b36-491c-84a8-8de453117831 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0db0b760-b4b7-41d2-a6cb-dbd1c8be4a01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:26.512288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:26.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:26.513865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:26.517937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 32/71] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:26.519367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.519737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.520034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.527488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 40/72] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:26.529445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.529744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.530028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.538716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:26.540262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:26.548000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:26.549068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:26.611390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 33/73] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:26.612329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.628935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 41/74] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:26.628935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.717199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf809a-522b-43b2-8544-1f63ef2393f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.731228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.749518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:26.756087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b19572-c3be-4fb0-8ba1-49cf00a4a144 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.767884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.769268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.769756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.770661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.771851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:26.772357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:26.775405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:26.792287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 34/75] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:26.793496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.800807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.802305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.802869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.803348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.820264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:26.822086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:26.823951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 42/76] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:26.825076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.915772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc809d-9648-4413-b8ef-7362e7a683da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.930773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a2447-d78b-4208-97f8-7742353041fe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:26.949135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.949765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.950049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.950323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.960968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:26.964993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:26.965659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:26.965947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:26.966200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:26.967065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:26.967881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:26.968482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:26.968482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:26.968609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 35/77] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:26.968609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.969062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:26.970691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 43/78] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:26.971507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:26.980933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.026372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.073131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa8f9c-0efa-4923-9a45-422e35bfa1d3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.074115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008f93b-201b-4e13-8e9b-b83de27ff9a5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.106345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.106978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.107261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.107541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.108650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.110142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.111769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 44/79] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.112528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.115736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.118270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.118270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.118485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.185272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.185959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.186737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.210524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60719a5-2f34-4cfa-ad48-1d1ae0b573fe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.239805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.245899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.246539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.246823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.247219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.259475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:27.265698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:27.266586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:27.267346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 45/80] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:27.268007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.291461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 36/81] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:27.292524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.366367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a5661-ec46-466c-8ebc-5627fa0c726d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.389157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9a0f-e9f8-4257-a116-c04d9fd8ea72 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.399244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.399852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.400137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.400412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.401071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:27.401410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.402807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.412076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 46/82] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.414651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.428872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.430134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.430411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.430681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.451982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.497987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.499383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.513399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2a298-9472-465b-b088-0d67847a88d7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.547355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.547963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.548247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.548523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.549206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.550646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.571439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 47/83] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:27 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:27.582604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.598636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 37/84] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:27.599296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.651229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.665748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.704813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c90d7c-9b9c-44d3-9658-be5e123bc40f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.717622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4406d98-4e48-42f3-8c2b-4188d761e824 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.750924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.753501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.753501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.753501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.753501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:27.753501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.754562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.763311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 48/85] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.764163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.769642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.770405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.770791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.771183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.772684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.774598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.777637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 38/86] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.777637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.887697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da72cd4-0435-43ef-a705-9a99e2d69c72 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.903710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e10f9-506f-4355-907d-f25e8314d947 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:27.905339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.930717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.931109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.931460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.931731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.933325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.933665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:27.934940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.936606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 49/87] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.938134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:27.954316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:27.955423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:27.955971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:27.956444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:27.957918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:27.959708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:27.962701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 39/88] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:27.963709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.055908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d13c7-cbb7-404b-a4f6-48fdd5b20fc7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.080560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2f877-f901-4b5b-b44b-ad0136f94e23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.091076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.091722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.092005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.092283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.093481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.094931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.101441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 50/89] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:28.101441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.132377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.133499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.133887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.134269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.135826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:28.136249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.139082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.147282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 40/90] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:27 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:28.148931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.157658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:28.223741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8878768-f3d3-46f2-bcd4-134190fd07b1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.240249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:28.291589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.292229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.292568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.292863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.333966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb63b43-06d3-4330-bc1b-5d120eec9327 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.379644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.380261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.380547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.380819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.381593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=82706f28e6f34da4b5745e5a95638706) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:28.381902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.383380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.389210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 41/91] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/role_assignments?user.id=82706f28e6f34da4b5745e5a95638706&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:28.394048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.401228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.402869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.459464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:28.522079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 51/92] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:28.529150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.602022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d17b-f5ea-40b9-980f-91591e782dc7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.621495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:28.653249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.653883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.654171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.654456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.674974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97b889-f48d-4488-b8d7-c1ce9bfbc48c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.710827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.711467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.711751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.712028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] PATH_INFO: `/v3/projects/920ee17361b842c0976a13ffbd1a55a8` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.713196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] RBAC: Authorizing `identity:get_project(project_id=920ee17361b842c0976a13ffbd1a55a8)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.714664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.716630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 52/93] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/projects/920ee17361b842c0976a13ffbd1a55a8 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:28.721997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:28.764945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.774135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:28.774899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.880161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc982b87-f89a-47b2-8e8b-c6ea5302843f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:28.956845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:28.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:28.957791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:28.958076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:28.958877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=920ee17361b842c0976a13ffbd1a55a8, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:28.959198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:28.963320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:28.975816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 42/94] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:28.979159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 53/95] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=920ee17361b842c0976a13ffbd1a55a8&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:28.981688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:29.017859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:29.129635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883cd4b-0baf-42bf-85de-ed0f8cdf0549 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:29.181304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:29.181869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:29.182151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:29.182426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:29.189997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:29.191510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e6441-6f52-4126-9600-ab639e09243b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:29.193320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 43/96] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:28 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:29.199964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:29.328258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:29.349799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ba141-5adf-4650-b6cf-7cdc37c8d3d7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:29.392625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:29.393287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:29.393577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:29.393859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:29.394607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:29.394922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:29.396395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd946e-df35-4af3-8f3b-c650ea61e6a5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:29.405937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 54/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:29 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:29.922291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026790-5bd6-463d-9715-71b53ddfeb12 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:29.923320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026790-5bd6-463d-9715-71b53ddfeb12 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:29.923812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4026790-5bd6-463d-9715-71b53ddfeb12 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:29.925680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 44/98] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:29.931860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567aad9c-d1ed-4b7e-bf84-95e77d58314b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:29.932783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567aad9c-d1ed-4b7e-bf84-95e77d58314b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:29.933288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567aad9c-d1ed-4b7e-bf84-95e77d58314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:29.958270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-567aad9c-d1ed-4b7e-bf84-95e77d58314b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:29.959870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-567aad9c-d1ed-4b7e-bf84-95e77d58314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:30.039428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 55/99] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:30.420989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026790-5bd6-463d-9715-71b53ddfeb12 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:30.532964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:30.555279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4026790-5bd6-463d-9715-71b53ddfeb12 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:30.602056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:30.603059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:30.603538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:30.604003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:30.618221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:30.622503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e53740-df8c-48eb-842e-eaed8ed6aeb6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:30.622794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e53740-df8c-48eb-842e-eaed8ed6aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:30.623074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e53740-df8c-48eb-842e-eaed8ed6aeb6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:30.624181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:30.625015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:30.625820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:30.626403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 56/100] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:30.628098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 45/101] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:30.631320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:30.631320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:30.631320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:30.634002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:30.666112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:30.667305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:30.786433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 57/102] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:30.876877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ea62e-274c-4010-8d9f-e39c549a046d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:30.932671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:30.933318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:30.933596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:30.933874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:30.934647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:30.934960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:30.936379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:30.951549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 46/103] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:30.957055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:30.971536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.088055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfcd8-93bb-413f-9832-8a24fc21e871 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:31.122559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:31.123419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.123838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.124249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] PATH_INFO: `/v3/users/73f9ea9253754d0699c5900b6d652fd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.148928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] RBAC: Authorizing `identity:get_user(user_id=73f9ea9253754d0699c5900b6d652fd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:31.150653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:31.152487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 58/104] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:30 2022] GET /identity/v3/users/73f9ea9253754d0699c5900b6d652fd1 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:31.162283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:31.177838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.186561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:31.249977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.289352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755e433-3d6a-44f5-bd2c-b714065aae64 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:31.324247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:31.325189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.325668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.326114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.327375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:31.328967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7873e87-9be2-4de6-a8e9-69cd0fe83d03 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:31.330997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 59/105] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:31.336629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1312-95bd-4d07-9e22-0315e0489a74 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.337278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1312-95bd-4d07-9e22-0315e0489a74 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.337707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f1312-95bd-4d07-9e22-0315e0489a74 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.339228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 60/106] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:31.340072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1312-95bd-4d07-9e22-0315e0489a74 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:31.344087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d91f74-179b-4c3d-86b3-a7d9280437a7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:31.406276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:31.407191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.407628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.408049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.447571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f1312-95bd-4d07-9e22-0315e0489a74 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:31.485559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.491435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:31.492481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.493953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.494402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.495833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:31.498682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2b06c-922b-4a69-976f-f461a5aed6f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:31.540500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:31.542554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:31.544283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 61/107] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:00:31 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:31.549141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd996c-a753-45b9-aebb-8f1dca5a6306 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.549141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd996c-a753-45b9-aebb-8f1dca5a6306 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.549141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd996c-a753-45b9-aebb-8f1dca5a6306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.559622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.572338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c8bd996c-a753-45b9-aebb-8f1dca5a6306 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:31.573848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c8bd996c-a753-45b9-aebb-8f1dca5a6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:31.662217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 62/108] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:31.664325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba777f-cc59-44c2-bbe5-df1b54f967d6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.664905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba777f-cc59-44c2-bbe5-df1b54f967d6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.665211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba777f-cc59-44c2-bbe5-df1b54f967d6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.667032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 63/109] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:31.672827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.673377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.673659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.696965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:31.698144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:31.783135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 64/110] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:31.815942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 47/111] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:31.822114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:31.881850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:31.938593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1273de-6713-4aee-b6f6-2fd7da79ecb8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:31.974941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:31.976270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:31.976695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:31.977135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:31.978425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:31.980060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:31.981942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 65/112] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:31 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:31.987910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.082428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.138207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f10ad-6db4-4c33-a3a8-c357aad8906e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.146383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:32.181409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.182625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.183104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.183577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.185301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=73f9ea9253754d0699c5900b6d652fd1) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:32.185640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:32.187114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4b56-1415-4437-a886-c65dbceb6b71 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:32.195930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 48/113] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:31 2022] GET /identity/v3/role_assignments?user.id=73f9ea9253754d0699c5900b6d652fd1&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=60751164de814fd294ca92d31ff8170e => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:32.197905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f469-73f1-48b6-b9d8-2b070a3808ae None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.198193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f469-73f1-48b6-b9d8-2b070a3808ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.198470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77f469-73f1-48b6-b9d8-2b070a3808ae None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 49/114] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb90c-26d2-438a-a81d-99a94aa93d31 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb90c-26d2-438a-a81d-99a94aa93d31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fb90c-26d2-438a-a81d-99a94aa93d31 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 50/115] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:32.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb90c-26d2-438a-a81d-99a94aa93d31 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.237075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ae7eb-9083-412c-a078-08b8d947dcb4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.258790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:32.292254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.293338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.293836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.294308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.310850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eddb0b75-f9f5-4201-b465-e1355b6288dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:32.318717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:32.319706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:32.321028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 66/116] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:32.322439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.324011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.324513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.335762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fb90c-26d2-438a-a81d-99a94aa93d31 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.349659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:32.351067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:32.375946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.376854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.377356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.377777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.391283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-00c1801e-97db-402d-98e8-e8b590c49cb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:32.397851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:32.398986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:32.399876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:32.401741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 51/117] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:32.403236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.403708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.404162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.423664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:32.425198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:32.441213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 67/118] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:32.442845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.469270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:32.548994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 52/119] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:32.551184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.638052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dd26d-7865-4113-b27a-05e9fb5ec9b4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.639579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:32.696947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.697596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.697875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.698147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.698886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:32.699123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:32.700513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:32.715328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 68/120] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:32.720761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.723338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76401de-7637-4e33-b870-dfc0d9b04057 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.804780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.805433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.805718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.806002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.806682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:32.806997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:32.808391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:32.839025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 53/121] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:32.844551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:32.878360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:32.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c854e9-a240-4a84-aad4-2315def5d8d3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.976779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa9a1b3-ca05-4620-ab94-dad319b00240 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:32.980566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:32.981290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:32.981635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:32.981978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:32.983210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:32.984931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:32.986797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 69/122] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:32.987636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.017343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.018310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.018875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.019160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.020469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.020854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.022290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.025167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 54/123] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.025856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.100345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df060-eba6-4507-83b5-26af80e740d2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.123076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.131818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a817e0-3d33-4343-9647-53336c7ed2a8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.137806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.138438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.138717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.138992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.151116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:33.156700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:33.159323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 70/124] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:33.161233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.168888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.173140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.173140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.173140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.180694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:33.186014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:33.188277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 55/125] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:33.190320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.259193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.268939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955893c-ed8a-4fe8-8266-fa3cbc88ab9e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.291516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc41fc-520e-4e69-9067-9fc949c986b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.312061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.312905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.313391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.313802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.315186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.316817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.319149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 71/126] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.320702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.326838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.327484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.327770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.328051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.329216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.330685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.332475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 56/127] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.333206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.353461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.435323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4dfb2-fd43-4b3c-a5c5-893d83cf52b2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.436590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22ba42-9c10-4970-8299-0abdaec45016 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.472573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.473329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.473623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.473902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.474303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.474574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:33.474883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.475441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.475940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.476301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.476555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.477623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:33.478395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.480974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.485266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 57/128] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.486196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.491023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 72/129] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.492202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.521390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.572388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.593317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9ca4-c552-40b7-8de3-b715203bb8d0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.611175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c1c69-9379-4d5f-9a13-b3f03e7cc80f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.628241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.628917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.629250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.629536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/7c987b3959274c0893cd501e70f6d998/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.631476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, user_id=7c987b3959274c0893cd501e70f6d998, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.633585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.650412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 58/130] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:33 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/7c987b3959274c0893cd501e70f6d998/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:33.651272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.651393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.652301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.652753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.653237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/82706f28e6f34da4b5745e5a95638706/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.656959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, user_id=82706f28e6f34da4b5745e5a95638706, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.658825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.683708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 73/131] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:33 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/82706f28e6f34da4b5745e5a95638706/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:33.685445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.807204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:33.833083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af045495-c13f-4afc-8221-724ca8540cc5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.862419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33d1b-1d16-4856-9a4d-f35c12559fa3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:33.902781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.903758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.904140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.904511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.906666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.908358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.910414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 74/132] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.918441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:33.919336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:33.919761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:33.920177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:33.923371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:33.924645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.925302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:33.927133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 59/133] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:33.933234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:33.951875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:34.070557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40831870-fa9e-473f-ac88-5359a3a8f968 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:34.125458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bae45-67e8-484c-888f-0ca7db767dea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:34.154388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:34.155068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.155350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.155633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.156810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:34.158321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e59112-de63-4a2b-b64f-fd2ee5af15aa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:34.160146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 75/134] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:34.161079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ed19-9269-4a50-9137-cb0d497b8b15 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.161480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ed19-9269-4a50-9137-cb0d497b8b15 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.161669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3ed19-9269-4a50-9137-cb0d497b8b15 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.162869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 76/135] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:34.167765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3ed19-9269-4a50-9137-cb0d497b8b15 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:34.197273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:34.207786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:34.208435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.208717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.208994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] PATH_INFO: `/v3/projects/920ee17361b842c0976a13ffbd1a55a8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.210358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] RBAC: Authorizing `identity:get_project(project_id=920ee17361b842c0976a13ffbd1a55a8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:34.212592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526a41-f9fb-406d-97f7-ef74a3c3d808 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:34.215739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 60/136] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:33 2022] GET /identity/v3/projects/920ee17361b842c0976a13ffbd1a55a8 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:34.216874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab378-c65d-47e1-80e4-b7994d09767b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.221956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab378-c65d-47e1-80e4-b7994d09767b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.222274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab378-c65d-47e1-80e4-b7994d09767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.257554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68fab378-c65d-47e1-80e4-b7994d09767b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:34.258780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68fab378-c65d-47e1-80e4-b7994d09767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:34.338830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3ed19-9269-4a50-9137-cb0d497b8b15 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:34.381002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:34.382256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.382757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.383256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/7c987b3959274c0893cd501e70f6d998/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.385954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, user_id=7c987b3959274c0893cd501e70f6d998, role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:34.387619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:34.410137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 77/137] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:34 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/7c987b3959274c0893cd501e70f6d998/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:34.411494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:34.417341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 61/138] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:34.511558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d4bb-617b-4d22-931d-13aa20b8bf3c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.511847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d4bb-617b-4d22-931d-13aa20b8bf3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.512130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d4bb-617b-4d22-931d-13aa20b8bf3c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.513776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 62/139] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:34.535234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d55743-3080-4651-9ed7-872443f1272d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.535671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d55743-3080-4651-9ed7-872443f1272d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.536052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d55743-3080-4651-9ed7-872443f1272d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.548846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:34.563342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:34.580616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4022d-543b-4d7f-ba2d-a1b77233b1fe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:34.586811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68d55743-3080-4651-9ed7-872443f1272d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:34.588050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68d55743-3080-4651-9ed7-872443f1272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:34.651315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:34.652358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:34.652856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:34.653352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] PATH_INFO: `/v3/projects/920ee17361b842c0976a13ffbd1a55a8/users/7c987b3959274c0893cd501e70f6d998/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:34.655598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] RBAC: Authorizing `identity:create_grant(project_id=920ee17361b842c0976a13ffbd1a55a8, user_id=7c987b3959274c0893cd501e70f6d998, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:34.657344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:34.679760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 78/140] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:34 2022] PUT /identity/v3/projects/920ee17361b842c0976a13ffbd1a55a8/users/7c987b3959274c0893cd501e70f6d998/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:34.732049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 63/141] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:35.009578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:35.140053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9942b-24af-417e-97b6-3e72cb277f74 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:35.182399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:35.183097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:35.183386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:35.183668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:35.184834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:35.186325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:35.188112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 79/142] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:35.193660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d55743-3080-4651-9ed7-872443f1272d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:35.313849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d55743-3080-4651-9ed7-872443f1272d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:35.323006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:35.333493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:35.383809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:35.384438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:35.384722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:35.385003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:35.385705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:35.387170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:35.406486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:35.433601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 64/143] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:35 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:35.502473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d56056-3542-40a5-8949-d0977bc5b6e6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:35.545623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:35.546287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:35.546569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:35.547039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:35.565158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:35.570645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:35.571485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:35.572241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 80/144] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:35.578779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:35.705250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dd6ac-2d05-486c-9cfe-7965291b45bd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:35.743097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:35.743774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:35.744065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:35.744346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:35.745027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:35.745357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:35.746822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:35.756029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 65/145] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:35.761721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:35.854680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:35.903185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10861376-7969-4d64-96a4-001f00e3aa74 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:35.948542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:35.949568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:35.950043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:35.950506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] PATH_INFO: `/v3/users/73f9ea9253754d0699c5900b6d652fd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:35.951863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] RBAC: Authorizing `identity:get_user(user_id=73f9ea9253754d0699c5900b6d652fd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:35.953558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:35.955353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 81/146] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:35 2022] GET /identity/v3/users/73f9ea9253754d0699c5900b6d652fd1 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:35.961808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:36.044111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:36.092752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbb3fa-3d66-493b-b964-3185ab34fb18 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:36.130076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:36.130643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.130928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.131234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.132398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:36.133990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8248888-496b-4373-9115-25134b5d0224 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:36.138046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 66/147] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:35 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:36.141486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:36.283098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc32b0-bd20-4acc-b3dd-2080e92e238a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:36.321614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:36.322281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.322561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.322833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/73f9ea9253754d0699c5900b6d652fd1/roles/60751164de814fd294ca92d31ff8170e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.324776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, user_id=73f9ea9253754d0699c5900b6d652fd1, role_id=60751164de814fd294ca92d31ff8170e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:36.326267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c933f-a696-4d1d-b7c3-09bb3c960f52 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:36.328552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:36.354556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 82/148] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:36 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/users/73f9ea9253754d0699c5900b6d652fd1/roles/60751164de814fd294ca92d31ff8170e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:36.388852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d7d0-861a-45f8-9809-0e331e63964b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.389358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d7d0-861a-45f8-9809-0e331e63964b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.389358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034d7d0-861a-45f8-9809-0e331e63964b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.391187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 67/149] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:36.396842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200fd2-2245-41d3-bc09-1c565b7c9346 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.397147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200fd2-2245-41d3-bc09-1c565b7c9346 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.397463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd200fd2-2245-41d3-bc09-1c565b7c9346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.422850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd200fd2-2245-41d3-bc09-1c565b7c9346 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:36.424146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd200fd2-2245-41d3-bc09-1c565b7c9346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:36.505028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 83/150] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:36.519074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ecb9-94b6-4f6e-a735-257bc580ab78 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.519366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ecb9-94b6-4f6e-a735-257bc580ab78 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.519653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196ecb9-94b6-4f6e-a735-257bc580ab78 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.521167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 68/151] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:36.526975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe781-6cf2-4732-a4b4-451c55ea844a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:36.527261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe781-6cf2-4732-a4b4-451c55ea844a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:36.527534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efe781-6cf2-4732-a4b4-451c55ea844a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:36.546768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-98efe781-6cf2-4732-a4b4-451c55ea844a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:36.547972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98efe781-6cf2-4732-a4b4-451c55ea844a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:36.629223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 84/152] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:36.891177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196ecb9-94b6-4f6e-a735-257bc580ab78 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.010068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196ecb9-94b6-4f6e-a735-257bc580ab78 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.012352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65519c-30cb-417e-abcf-65d8ab0b3e48 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.012634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65519c-30cb-417e-abcf-65d8ab0b3e48 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.012902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65519c-30cb-417e-abcf-65d8ab0b3e48 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.014273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 85/153] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:37.022979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.023231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.023449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.058017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.058917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:37.059832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:37.080566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.081373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.081714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.082135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.095835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:37.102685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:37.103629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:37.104403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 69/154] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:37.105028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.188167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.224074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea409a4-781a-4166-859c-d4d1ca959a92 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.226968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 86/155] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:37.228067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.282791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.283458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.283741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.284017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.318085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:37.324082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:37.325666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:37.327003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 70/156] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:37.332370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.397464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.435808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806a49b-d13d-4c0d-9bc1-6fc01fd4fd09 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.480565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.481200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.481478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.481748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.482418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:37.482716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:37.484090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:37.493777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 87/157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:37.499957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.503501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac36b-0b37-4582-9faa-1a73bc6f619a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.555065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.555243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.555611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.555890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.556164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.556846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:37.557215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:37.558583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:37.572014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 71/158] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:37.577322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.616197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ed62d-40d8-4705-9397-1ad5ec03028f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.656966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.657638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.657914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.658186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] PATH_INFO: `/v3/users/82706f28e6f34da4b5745e5a95638706` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.659507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] RBAC: Authorizing `identity:get_user(user_id=82706f28e6f34da4b5745e5a95638706)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:37.661463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:37.663319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 88/159] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/users/82706f28e6f34da4b5745e5a95638706 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:37.669841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:37.786932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212400fc-e088-4ad6-8e25-efabcfbd771d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.808144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.837214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.838067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.838353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.838629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.853073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4fc98-4460-427f-80a9-b8678f1e3041 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:37.859507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:37.934025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:37.935691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:37.986731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:37.987385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:37.987662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:37.987932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:37.989077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:37.990601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:37.993081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 72/160] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:37.997995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.086978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 89/161] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:38.088596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.114433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:38.187422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e51f-9d4e-4208-a045-336ca692f6d4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:38.251476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:38.252491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.252932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.253361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.270942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65e54b-08fc-4ff7-bd07-69ee347eb6d1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:38.277138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8a721580-11ac-4ad6-86ce-4e4db3df27db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:38.280514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:38.281483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:38.282245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:38.284958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 73/162] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:38.290699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c3f0-2b2a-4c12-9a90-811c04edb323 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.291578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c3f0-2b2a-4c12-9a90-811c04edb323 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.292033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40c3f0-2b2a-4c12-9a90-811c04edb323 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.293829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 74/163] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:38.296633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40c3f0-2b2a-4c12-9a90-811c04edb323 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.344745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:38.349406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.349860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.350305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.367730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:38.465939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40c3f0-2b2a-4c12-9a90-811c04edb323 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:38.468158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:38.469663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ca9a8-5788-4d4f-b80d-7d13e805a382 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:38.504621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:38.538413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:38.539053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.539333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.539607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.541503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:38.542985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:38.544638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 75/164] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:38 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:38.545434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.664833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 90/165] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:38.666030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.666322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.666600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.703619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:38.704809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:38.721820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fcb9b-57d4-4a4c-aee6-3ecc89354fd8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:38.724984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:38.790730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:38.791364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:38.791643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:38.791914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:38.792587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:38.792891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:38.794336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:38.808587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 76/166] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:38.809457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.842473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 91/167] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:38.843384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:38.914004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:38.996238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7db57-5219-426f-8492-22e112fe42d4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.043312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74c06b-1f41-4d11-a5d7-afcb2a03721b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.067810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.068602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.068892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.069193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.071949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:39.073796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:39.075476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 92/168] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:38 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:39.076255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.118065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.119129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.119617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.120084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.121034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=82706f28e6f34da4b5745e5a95638706) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:39.121562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:39.123587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:39.136057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 77/169] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:38 2022] GET /identity/v3/role_assignments?user.id=82706f28e6f34da4b5745e5a95638706&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:39.137180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.189905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:39.252981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2039-6df8-4970-96b9-cb3cfa9dfa8b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.289501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:39.310141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.310779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.311058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.311329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.321165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:39.321165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:39.321725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 93/170] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:39.323783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19636f2d-533c-4cbf-9b51-69334b789ff0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.327686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.376283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.377349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.377834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.378309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.379269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:39.379786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:39.385714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:39.396750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 78/171] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:39.454280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.473179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c316c-f10d-4a62-94d3-58b67758f591 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.519144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.519880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.520057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.520328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.607219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:39.648964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:39.650166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:39.651900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:39.668405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ad9e3-cb6c-402d-bfa6-b2d01d2976f2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.730816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:39.731719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:39.732295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:39.732714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:39.754712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:39.761283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:39.762269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:39.763055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:39.769893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 79/172] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:39.780831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.839592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 94/173] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:39.845469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:39.980093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346d5a0-4e36-4114-9b7b-0382f2f3cae7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:39.995718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.022151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.023604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.024033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.024725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.025556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:40.026229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:40.028180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cda1ef-6a69-4e22-b6db-51793729bfc2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:40.031795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294e924-e6c6-4348-99f1-58d442fd312e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.039603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 80/174] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:40.041109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878c7d-f431-4e89-92bf-7a45bd023761 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.041568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878c7d-f431-4e89-92bf-7a45bd023761 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.041964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19878c7d-f431-4e89-92bf-7a45bd023761 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.046584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 81/175] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:40.047899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0ef3-dc5c-4739-8c29-c3969c4d995c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.048343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0ef3-dc5c-4739-8c29-c3969c4d995c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.049236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0ef3-dc5c-4739-8c29-c3969c4d995c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.051163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 82/176] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:40.052214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c0ef3-dc5c-4739-8c29-c3969c4d995c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.083585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.084365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.084745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.085172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.086422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:40.088039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be642d-aa9a-44cc-b257-e75a9a7e2339 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:40.089786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.095110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 95/177] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:39 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:40.097494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a27bf-f051-41ef-9a89-1bc8b1c2621c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.097771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a27bf-f051-41ef-9a89-1bc8b1c2621c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.098038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a27bf-f051-41ef-9a89-1bc8b1c2621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.118943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-276a27bf-f051-41ef-9a89-1bc8b1c2621c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:40.120130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-276a27bf-f051-41ef-9a89-1bc8b1c2621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:40.180069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c0ef3-dc5c-4739-8c29-c3969c4d995c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.193897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 96/178] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:40.195489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.195980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.196375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.215516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:40.216985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:40.234134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.235123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.235474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.235820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] PATH_INFO: `/v3/users/7c987b3959274c0893cd501e70f6d998` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.246628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] RBAC: Authorizing `identity:get_user(user_id=7c987b3959274c0893cd501e70f6d998)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:40.248522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:40.251206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 83/179] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/users/7c987b3959274c0893cd501e70f6d998 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:40.252670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.296842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 97/180] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:40.298535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.309961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.415887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.452778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec87b20-4ba8-4ef0-9e4b-f5f07572ec60 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.466147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6b7873-1a12-4198-9008-16d0ff89043c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.506181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.506795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.507674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.507954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.508644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:40.508949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:40.510435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf8a37-fc77-4037-a477-2fa776f6226b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:40.523899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 84/181] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:40.525138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8eb49-c3a1-4738-affc-e79345c6a619 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.525426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8eb49-c3a1-4738-affc-e79345c6a619 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.525699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8eb49-c3a1-4738-affc-e79345c6a619 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.527533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.528153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.528431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.528704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.538288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 85/182] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:40.538748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.539036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.539311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.581335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:40.582976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:40.652503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:40.654750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101916a9-d76a-4a19-ab0d-62f46fd5c258 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:40.668732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 86/183] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:40.669285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.669926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.757219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.786699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5a3e9-81a4-4a04-80ce-0cab92ad044f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.792088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 98/184] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:40.792088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9c7c-8af4-4726-baf5-172aeae0ae01 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.792088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9c7c-8af4-4726-baf5-172aeae0ae01 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.792344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c9c7c-8af4-4726-baf5-172aeae0ae01 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.798026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 99/185] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:40.798666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9c7c-8af4-4726-baf5-172aeae0ae01 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.836583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.837968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.837968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.837968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.853997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:40.858895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:40.865293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 87/186] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:40.867028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:40.927136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c9c7c-8af4-4726-baf5-172aeae0ae01 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.937631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:40.962184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:40.962814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.963089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.963360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.973355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:40.974081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffe79a-5530-4b95-8163-da1e31992343 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-87c1c3c3-66e7-41f1-91fe-6be3455a0889 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:40.978756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:40.979886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:40.980677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 100/187] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:40.981639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:40.981639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:40.981639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:40.999226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:41.000398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:41.009083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.009742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.010017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.010284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] PATH_INFO: `/v3/projects/920ee17361b842c0976a13ffbd1a55a8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.011387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] RBAC: Authorizing `identity:get_project(project_id=920ee17361b842c0976a13ffbd1a55a8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.012818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.014459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 88/188] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:40 2022] GET /identity/v3/projects/920ee17361b842c0976a13ffbd1a55a8 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.015208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.021440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.067482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 101/189] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:41.070569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.120570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af2f55-bd23-4e1d-a473-0cc951397c7e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.156950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.157584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.157863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.158261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.158925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:41.159229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.160738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.166868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.170781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0c9db-b64c-4404-834d-f9733df8a355 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.175336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 89/190] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.175336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.212140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.212759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.213030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.213323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.223519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:41.228851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:41.229901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 102/191] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:41.230682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.243826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.285313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726a61d-4833-4f85-8014-bb1791c087ea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.319401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.320032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.320311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.320584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.321275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:41.321579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.323037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.331308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 90/192] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.336467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.342428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde006be-6c72-4844-bb84-6566c040c9cf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.375431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.375957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.376233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.376510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.377239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=920ee17361b842c0976a13ffbd1a55a8, user.id=7c987b3959274c0893cd501e70f6d998) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:41.377544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.378966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.384490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 103/193] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/role_assignments?user.id=7c987b3959274c0893cd501e70f6d998&scope.project.id=920ee17361b842c0976a13ffbd1a55a8&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.385291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.428782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.439859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bcf2d-cab1-4e1f-a2bb-d1f68e9a0e68 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.470294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.480603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.481291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.481568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.481839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.502055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.503690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.510888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 91/194] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.510888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.523211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad35560-8d14-4801-8919-995651ed5887 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.565151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.565377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.565739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.565898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.567081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:41.567081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.569143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.577983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 104/195] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.578802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.627048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017eb4-94f1-4ec7-845f-b06000746fcd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.661093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.661739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.662015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.662287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.663407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.664230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.664870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.666566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 92/196] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.667333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.682799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34472f-3834-4c2b-9440-fa8976fc13e6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.716773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.717523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.717875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.718225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] PATH_INFO: `/v3/users/73f9ea9253754d0699c5900b6d652fd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.719360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] RBAC: Authorizing `identity:get_user(user_id=73f9ea9253754d0699c5900b6d652fd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.720810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.721352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.722454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 105/197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/users/73f9ea9253754d0699c5900b6d652fd1 => generated 289 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.723213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.776071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ba5a5-60f9-4489-8aec-09294a91647d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.811586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.812289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.812567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.812839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.820893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a00ea-0f66-421d-870e-c863f1d8a87e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.853732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.854348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.854622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.854901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.856010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.857481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.863563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 106/198] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:41.863563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.879907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.881341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:41.883246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.932112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:41.959740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698fc79-cb9a-46ae-8cef-3258123bbf1a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:41.979031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 93/199] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:41.979859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:41.993832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:41.994439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:41.994721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:41.994990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:41.995635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:41.997072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.017906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 107/200] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:41 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:42.017906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.089212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c561d-8202-4441-81a5-d2c09165e679 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.105444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.133888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.134747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.135141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ed27b-578a-4d16-a4ba-507e2e65e72c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.135357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.135771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.164907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.173278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.173724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.174002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.174275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.229201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.230699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.265599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.269100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.360150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.367251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 94/201] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:42.368449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.394021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 108/202] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:42.395269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.436215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.497896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324ae8-0f4c-4b4a-8f95-9aee47d4685d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.521715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f000fd5-3efd-4c75-9d37-61e1ead38cd7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.546350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.547886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.548483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.548968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.552320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.555643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.559152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.559650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 95/203] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:42.560769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.565416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.565416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.565663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.566897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.568377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.570428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 109/204] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:42.571705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.620222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.679684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a580b-66b3-451a-8cfb-74abfa36d9b0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.680261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d64f374-3351-4b19-a960-15af981c621e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.701185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.722091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.722723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.723001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.723286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.724513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.726655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.728173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.728422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 96/205] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:42.729037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.729442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.729811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.730714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=82c3454ad68f4f3482b8a5f4e047f8d9) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:42.731132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.732729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.733916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.739175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 110/206] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/role_assignments?user.id=82c3454ad68f4f3482b8a5f4e047f8d9&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:42.740156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:42.857791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407d7f0-f241-406d-8c1b-a438da6b5ee1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:42.879419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:42.904533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:42.905886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:42.906166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:42.906619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:42.907372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=82c3454ad68f4f3482b8a5f4e047f8d9) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:42.907686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:42.914530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460b49e-d2f4-4e7c-8bcc-3630e8367458 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:42.921796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 111/207] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/role_assignments?user.id=82c3454ad68f4f3482b8a5f4e047f8d9&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:42.958519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec993f-c359-4bdf-a576-5cb2919f2271 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:43.004769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:43.005810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:43.006326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:43.006803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:43.008313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, scope.project.id=1750242301a84f42b1c4b15f4e538db9, user.id=73f9ea9253754d0699c5900b6d652fd1) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:43.008836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:43.010586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5b81d-c077-4dde-a8cf-0de762e1cf28 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:43.028446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 97/208] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:42 2022] GET /identity/v3/role_assignments?user.id=73f9ea9253754d0699c5900b6d652fd1&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=60751164de814fd294ca92d31ff8170e => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:44.169723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f8926-5779-429b-887c-dc8b3f6f2c01 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.170336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f8926-5779-429b-887c-dc8b3f6f2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.170336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f8926-5779-429b-887c-dc8b3f6f2c01 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.171877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 112/209] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:44.177309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b65da-51fe-450d-b3f6-b0991fe131a5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.177738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b65da-51fe-450d-b3f6-b0991fe131a5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.177940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b65da-51fe-450d-b3f6-b0991fe131a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.196728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-017b65da-51fe-450d-b3f6-b0991fe131a5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:44.197978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-017b65da-51fe-450d-b3f6-b0991fe131a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:44.250981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:44.276669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 98/210] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:44.336157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0486-96b0-4db4-8b0a-ee8a53b11415 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.336828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0486-96b0-4db4-8b0a-ee8a53b11415 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.337344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f0486-96b0-4db4-8b0a-ee8a53b11415 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.339304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 113/211] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:44.344730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.345019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.345334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.367722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:44.369526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:44.444744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 99/212] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:44.574302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0486-96b0-4db4-8b0a-ee8a53b11415 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:44.679849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f0486-96b0-4db4-8b0a-ee8a53b11415 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:44.717031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:44.717668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.717952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.718225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.725131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:44.729375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:44.734828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:44.735729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:44.736528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 114/213] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:44.743836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:44.794012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:44.840449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0a4a9-6f74-4696-a288-1b6ab46a5328 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:44.854937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:44.879159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:44.880007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.880291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.880557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.881844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:44.883818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66ae48-4711-4cd7-9252-fb2dd569aeb3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:44.883942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a9a0c-bbac-4ef0-be15-11a68d610259 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:44.885614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 100/214] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:44.886559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9679e-640b-46b3-93f9-e48a748471c7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.886844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9679e-640b-46b3-93f9-e48a748471c7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.887118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9679e-640b-46b3-93f9-e48a748471c7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.888441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 101/215] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:44.892964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9679e-640b-46b3-93f9-e48a748471c7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:44.928437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:44.929992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.930492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.930965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.931963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:44.932517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:44.933941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead8092-67bb-4b77-93dc-eb8ca2414679 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:44.948756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 115/216] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:44.949925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:44.950216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:44.950495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:44.968185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:44.969332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:45.001655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9679e-640b-46b3-93f9-e48a748471c7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.036195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.036806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.037085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.037391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.038044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.045310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 116/217] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:45.045310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.045549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6cec2-ce55-4c8e-a8c9-2b2ac0f568ee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.077572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 102/218] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:44 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:45.083267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.143588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c288c-f02f-439a-86b5-1b8ad03f9066 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.143931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c288c-f02f-439a-86b5-1b8ad03f9066 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.144141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c288c-f02f-439a-86b5-1b8ad03f9066 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.149726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 103/219] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:45.151448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.151737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.152015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.170935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31befc9-acf8-4be7-9b2e-33f1f40d5bf7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.171495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:45.172363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:45.207794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.222399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.223030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.224040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.224332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] PATH_INFO: `/v3/users/64f7fed227e4422c9c801977c7b5bab0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.246413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] RBAC: Authorizing `identity:get_user(user_id=64f7fed227e4422c9c801977c7b5bab0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.247911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.249896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 117/220] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/users/64f7fed227e4422c9c801977c7b5bab0 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:45.255693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.263095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 104/221] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:45.314773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.355300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.384379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb99b0b-792c-44b7-b404-22dbd86fcb7e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.439815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.439815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.439815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.439815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.448543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6463244-1118-4385-abe6-0a1661e2c24f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.498842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.500349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.501017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.501525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.516369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:45.524117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:45.525257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:45.526254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:45.528853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.529019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 105/222] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:45.532099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.536281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.574210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.620158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.640407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 118/223] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 07:00:45.641099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.675021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59900cf-bad0-46cf-8fe5-ec3fa008ebfb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.711522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.712142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.712422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.712696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.713405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:45.713707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.715108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.724004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 106/224] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:45.724929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.748563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db520115-bc0b-46d9-ad3f-ea4c02cafeca None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.786402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.787070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.787353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.787633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.799150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:45.804918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:45.806024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:45.806987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:45.808854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 119/225] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:45.809851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.831758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.844006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7f5b6-622e-4ad9-8f22-037a0b07a8a7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.860122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:45.884615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.885324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.885611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.885890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.887153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.888768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.890450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 107/226] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:45.896004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:45.935206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d4a66-5dd2-475d-9a80-cc4a50f49259 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:45.980917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:45.981729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:45.982356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:45.982726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:45.984822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:45.987555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:45.993298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 120/227] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:45.994200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.025347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da660-41a8-4209-9024-e23bd2b54806 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.061746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.062546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.062923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.063295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.064083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:46.064506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.066022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.075114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 108/228] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:46.076034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.096101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:46.108710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0e345-861b-42af-a025-31636a36f4bf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.119658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:46.144589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.145228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.145515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.145789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.146517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=64f7fed227e4422c9c801977c7b5bab0) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:46.146825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.148905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.160118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 121/229] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:45 2022] GET /identity/v3/role_assignments?user.id=64f7fed227e4422c9c801977c7b5bab0&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:46.161069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.189649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677a37e-c72c-415c-9bf5-f2b732c11922 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.231792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.232715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.232997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.233324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.234726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.236206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.239188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 109/230] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:46 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:46.245610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.350001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:46.368546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccd5d6-654e-4054-a7fd-d05b949f91fc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.406440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca854-8b30-40db-8838-70f54ff57052 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.408772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.409494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.409773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.410047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/82c3454ad68f4f3482b8a5f4e047f8d9/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.412906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, user_id=82c3454ad68f4f3482b8a5f4e047f8d9, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.414458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.435557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 110/231] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:46 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/82c3454ad68f4f3482b8a5f4e047f8d9/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:46.470102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.470828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.471276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.471569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.474723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.476390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.478516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 122/232] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:46 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:46.484313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.618866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98cddc-790b-4195-95fc-69c0138ca741 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.655477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.656160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.656442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.656712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.657881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.659339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ce675-f25f-43d3-a23b-1efcd4c38265 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.661165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 111/233] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:46 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:46.666851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:46.721143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:46.806730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb4f2c-97ff-4263-882e-eb7fd23e4c38 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:46.860169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:46.860844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:46.861143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:46.861423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/82c3454ad68f4f3482b8a5f4e047f8d9/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:46.866931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, user_id=82c3454ad68f4f3482b8a5f4e047f8d9, role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:46.868662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7ca75-9c4a-4099-875f-6277367e3acc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:46.896048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 123/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:46 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/82c3454ad68f4f3482b8a5f4e047f8d9/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:47.441609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92430-de6c-45c8-93d1-5819fd524e31 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:47.442116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92430-de6c-45c8-93d1-5819fd524e31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:47.442189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce92430-de6c-45c8-93d1-5819fd524e31 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:47.443804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 112/235] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:47.455091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b606f-eecd-42f7-b121-8d21e9fb8b0c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:47.455607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b606f-eecd-42f7-b121-8d21e9fb8b0c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:47.455689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b606f-eecd-42f7-b121-8d21e9fb8b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:47.482160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d91b606f-eecd-42f7-b121-8d21e9fb8b0c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:47.483378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d91b606f-eecd-42f7-b121-8d21e9fb8b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:47.570313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 124/236] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:47.611245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69e84-f5ad-4da5-a8d9-6a82116382c4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:47.611877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69e84-f5ad-4da5-a8d9-6a82116382c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:47.612349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d69e84-f5ad-4da5-a8d9-6a82116382c4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:47.614192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 113/237] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:47.620120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:47.620636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:47.621102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:47.649984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:47.651554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:47.692598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:47.747678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 125/238] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:47.947878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d69e84-f5ad-4da5-a8d9-6a82116382c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:47.965642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:48.063185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:48.080089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d69e84-f5ad-4da5-a8d9-6a82116382c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.137737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.138304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.138581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.138850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.164728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:48.172009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:48.172884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:48.173705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:48.181203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 114/239] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:48.185855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:48.197637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0662ae9-5ce7-455e-a6b7-135e2ab821c9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.246940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.247980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.248507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.249076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.250451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:48.252111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:48.254354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 126/240] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:48.259646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:48.290130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:48.317154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:48.319597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9b51-3b49-4376-9fec-074822425fa4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.365500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.366891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.367177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.367458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.368372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:48.368672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:48.371558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:48.394479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 115/241] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:48.401994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:48.464926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32ad3-1df6-4b94-9b66-d9a621c98481 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.529404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.530022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.530303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.530576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.531235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:48.532668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:48.571499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ea2ac-fbe2-4149-885b-53130949879a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.578439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 127/242] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:48 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:48.635064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.635812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.636092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.636411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] PATH_INFO: `/v3/users/85fb6b7fa0254f6694ac9f72062c761b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.639828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:48.655504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] RBAC: Authorizing `identity:get_user(user_id=85fb6b7fa0254f6694ac9f72062c761b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:48.657008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229151-1d6b-470e-be33-86861bea7d70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:48.659035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 116/243] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:48 2022] GET /identity/v3/users/85fb6b7fa0254f6694ac9f72062c761b => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:48.665466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:48.706621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ada6e5-a079-4c72-9b04-266c7703f757 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.706950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ada6e5-a079-4c72-9b04-266c7703f757 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.707172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ada6e5-a079-4c72-9b04-266c7703f757 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.708620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 117/244] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:48.715499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efa3b-7cfb-4201-8276-e2062c597a1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.715499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efa3b-7cfb-4201-8276-e2062c597a1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.715499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6efa3b-7cfb-4201-8276-e2062c597a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.743992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba6efa3b-7cfb-4201-8276-e2062c597a1e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:48.745232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba6efa3b-7cfb-4201-8276-e2062c597a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:48.814184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:48.837314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d53250-50eb-478d-bb29-1b92595b9561 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:48.840294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 118/245] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:48.875886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:48.876493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.876768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.877045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.935484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046089-0174-405c-a833-5a2b41753679 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.935966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046089-0174-405c-a833-5a2b41753679 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.936342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046089-0174-405c-a833-5a2b41753679 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.940627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 119/246] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:48.946584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:48.947056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:48.947343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:48.948444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:48.949985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:48.969047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:48.971929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:49.056270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.061641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 120/247] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:49.074262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 128/248] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:49.079858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.139724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.167147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.193979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd949ff-1e78-46b1-99ac-77980dd9a418 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.237428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.238408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.238878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.239345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.240669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:49.242430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:49.244951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 121/249] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:49.251152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.312272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72012ace-557d-4408-a44a-86854903bcdf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.352260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.353244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.353719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.354279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.363557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb244dc8-1f04-4305-8981-4e6806c79ba3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.367038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-20ae16f6-1e02-4346-9c7e-6926f308feb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:49.372388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:49.373486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:49.374451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:49.376667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 129/250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:49.378194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5dc52-26fa-4d10-97b0-d40f7bd55400 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.378637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5dc52-26fa-4d10-97b0-d40f7bd55400 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.379154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5dc52-26fa-4d10-97b0-d40f7bd55400 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.380545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 130/251] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:49.383069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5dc52-26fa-4d10-97b0-d40f7bd55400 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.389060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.401670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.403385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.403385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.403385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.405064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=85fb6b7fa0254f6694ac9f72062c761b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:49.405064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:49.406051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759ef14-4eba-49b1-96bf-fb047aeb280c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:49.413858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 122/252] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/role_assignments?user.id=85fb6b7fa0254f6694ac9f72062c761b&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=60751164de814fd294ca92d31ff8170e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:49.415137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.415685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.415966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.438999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:49.440941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:49.449300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.548563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5dc52-26fa-4d10-97b0-d40f7bd55400 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.579062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 123/253] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:49.580714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.607383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.607971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.608252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.608523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.609491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:49.609798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:49.611829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:49.628882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 131/254] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:49.635236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.764873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f28e9-8b41-4cfd-af81-cecdcaa13ec1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.812986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4984d-d348-452d-af19-b6aad7a89eee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:49.813556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.833473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:49.846793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.847579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.847930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.848317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.864836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:49.881821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:49.882734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:49.883597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:00:49.884420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:49.885185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:49.885185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:49.885185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] PATH_INFO: `/v3/users/64f7fed227e4422c9c801977c7b5bab0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:49.888458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 124/255] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:49.889243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] RBAC: Authorizing `identity:get_user(user_id=64f7fed227e4422c9c801977c7b5bab0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:49.889624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:49.890726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:49.892634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 132/256] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/users/64f7fed227e4422c9c801977c7b5bab0 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:49.898539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.041279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aba81c-4224-42a1-8864-3faf707aeafe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.071358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5aaeb-3811-4445-8bcf-55e321b98841 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.078760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.079680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.080177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.080732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.082152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.083856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.086524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 133/257] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:50.087557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.106952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.107560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.107839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.108117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.108938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:50.109273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.110950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.120652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 125/258] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:50.121412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.164394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:50.168638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:50.262387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52607b7-837a-4992-9e1a-f56c77972f60 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.281485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240c6b-3983-49c3-be40-06deb5e4561c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.323709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.324425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.324705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.324973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/64f7fed227e4422c9c801977c7b5bab0/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.327687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, user_id=64f7fed227e4422c9c801977c7b5bab0, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.329171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.353047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 126/259] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:50 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/64f7fed227e4422c9c801977c7b5bab0/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:50.353913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.355573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.356219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.356499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.358808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.380810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:50.390274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:50.391371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:00:50.392170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:50.392761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 134/260] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:00:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:50.398410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.574334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:50.581813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4ba17-7f92-45cf-94e6-078ef236f0f9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.645385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090864e-99fc-4e27-af0f-97e2b0dce4ce None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.650327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.651028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.651304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.651620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.652410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:50.656504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.658089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.660299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 127/261] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:50 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:50.666537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.715730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.716314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.716598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.716872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.717561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:50.717862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.719234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.734920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 135/262] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:00:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:50.742259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:50.858297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f111876-e997-4546-bdfe-d41a57cc3bc0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.898229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:50.898765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:50.899045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:50.899319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:50.959049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48138ca-1c4a-449e-97fb-7494e65f859a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:50.973155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:50.974647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:50.997619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:51.081596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 128/263] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:51.081869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:51.082527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.082814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.083092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] PATH_INFO: `/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.089374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] RBAC: Authorizing `identity:get_user(user_id=82c3454ad68f4f3482b8a5f4e047f8d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:51.090306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:51.090847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:51.092755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 136/264] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:50 2022] GET /identity/v3/users/82c3454ad68f4f3482b8a5f4e047f8d9 => generated 283 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:51.098861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:51.194547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:51.207987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bf021-5b6e-4cdb-a42f-aa212065de2d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:51.235675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:51.250058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:51.251018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.251522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.251993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.253368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:51.255023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:51.257880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 129/265] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:51 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:51.265687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:51.475408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e4c40-b96e-4b50-bdb5-433813e084e0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:51.520362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bed963-2c6c-482d-a86c-de5e6e7f30ef None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:51.549297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:51.550276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.550782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.551382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.646452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:51.647138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.647365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.647679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.648429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=82c3454ad68f4f3482b8a5f4e047f8d9) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:51.648740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:51.650330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1f3a6-c78c-4b70-948e-d95ec2761584 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:51.652366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:51.652768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:51.654458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aca8a3-ab98-4cce-aaea-5e80d3032090 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:51.671430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 130/266] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:51 2022] GET /identity/v3/role_assignments?user.id=82c3454ad68f4f3482b8a5f4e047f8d9&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:51.672784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e07b24-a42b-4f61-8064-aa8313a98e90 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.673073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e07b24-a42b-4f61-8064-aa8313a98e90 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.673384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e07b24-a42b-4f61-8064-aa8313a98e90 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.674959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 131/267] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:51.680628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d5fdb-89c9-4632-960f-16ce9895fd3f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.680945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d5fdb-89c9-4632-960f-16ce9895fd3f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.681198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d5fdb-89c9-4632-960f-16ce9895fd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.696906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:51.717826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba4d5fdb-89c9-4632-960f-16ce9895fd3f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:51.719393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba4d5fdb-89c9-4632-960f-16ce9895fd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:51.823754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 137/268] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:51.826635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d492-282b-41d7-bd33-4606356d7e91 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.826930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d492-282b-41d7-bd33-4606356d7e91 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.827206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91d492-282b-41d7-bd33-4606356d7e91 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.830620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 138/269] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:51.832682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91d492-282b-41d7-bd33-4606356d7e91 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:51.857318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 132/270] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:51.860208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:51.860500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:51.860776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:51.897760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:51.898957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:52.018645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91d492-282b-41d7-bd33-4606356d7e91 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.057556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.058471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 133/271] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:52.058738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.059203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.059702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.061367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.063093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.065210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 139/272] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:51 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.070651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.141538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.178434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.182462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.212983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19c00a-28d6-4c7a-861d-fcbe873e5bba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.256677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.257704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.258202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.258662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.259571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=82c3454ad68f4f3482b8a5f4e047f8d9) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:52.260067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.261722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5470e9b-10b3-4364-a45f-6e218f978713 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.268194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 134/273] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/role_assignments?user.id=82c3454ad68f4f3482b8a5f4e047f8d9&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.269726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef255-8c02-481e-b81f-53dd6a91f942 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.270219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef255-8c02-481e-b81f-53dd6a91f942 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.270730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ef255-8c02-481e-b81f-53dd6a91f942 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.272401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 135/274] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:52.278266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.278946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.279411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.316193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:52.318722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:52.328016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667ab93-d35f-4dae-8e77-28169f034bcc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.381375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.383012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.383511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.384023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.397254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:52.404159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:52.405096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:52.405948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:52.414321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 140/275] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:52.415555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.446414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 136/276] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:52.447422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.492757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.506245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.540626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9024017-9321-4090-ba4d-ad02940cb8fd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.558731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd86eeb-6d40-4302-b0e0-b8c339ede25f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.577448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.578327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.578776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.579267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.580212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:52.580738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.582360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.592483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 141/277] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.599645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.601771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.602509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.602836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.603171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.604484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.606541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.608537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 137/278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.613507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.712595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e059-c49f-409a-8ba9-42fed1a85e8c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.727013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49347fce-e9ac-450d-8159-8e7ea27ce9bc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.739061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.755109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.755940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.756325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.757443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] PATH_INFO: `/v3/users/62b8a180bb024afba321ab24f443d4ea` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.758274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:52.774186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.775145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.775495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.775769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.776540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.778005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.778269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] RBAC: Authorizing `identity:get_user(user_id=62b8a180bb024afba321ab24f443d4ea)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.779727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.786509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 142/279] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/users/62b8a180bb024afba321ab24f443d4ea => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.786509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.809248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 138/280] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:00:52 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:52.809248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:52.928085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c0f3a-a70c-45e5-84a9-bb7475b97be1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.932328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77a41a-af13-498b-9ce0-0ad2a8a4ee32 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:52.971473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.972087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.972367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.972641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.973401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, user.id=62b8a180bb024afba321ab24f443d4ea, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:52.973768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:52.975225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:52.980786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:52.981672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:52.982123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:52.982471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 139/281] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/role_assignments?user.id=62b8a180bb024afba321ab24f443d4ea&scope.system=all&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:52.982797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:52.995463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9fd8766e-614f-4f87-a577-a1da2d03a19b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:53.000650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:53.001657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:53.002451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 143/282] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:53.008525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.057957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:53.102858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d639e-581e-4824-a8e1-2d1938b93c10 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.103102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d639e-581e-4824-a8e1-2d1938b93c10 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.103502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d639e-581e-4824-a8e1-2d1938b93c10 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.104999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 144/283] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:53.110747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.111140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.111490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.133988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fe8a-cd31-46f6-b7e7-95dce140b383 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:53.137160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:53.137561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:53.179763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:53.180382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.180660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.180933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.181753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:53.182057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:53.183568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:53.183936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:53.195007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 140/284] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:53.200682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.225894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 145/285] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:53.315916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b420e57-2baf-48bd-8fda-7053832ee931 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:53.333618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:53.354259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:53.354974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.355344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.355760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] PATH_INFO: `/v3/users/85fb6b7fa0254f6694ac9f72062c761b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.356983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] RBAC: Authorizing `identity:get_user(user_id=85fb6b7fa0254f6694ac9f72062c761b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:53.358571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:53.360328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 141/286] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:53 2022] GET /identity/v3/users/85fb6b7fa0254f6694ac9f72062c761b => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:53.365973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.475062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ac916-f1b2-4a98-893c-9cac8f22fbe7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:53.512054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:53.512684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.512961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.513267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.514373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:53.515807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:53.517753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 146/287] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:53 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:53.523151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.613714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.668792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649397b6-02fd-4fc0-af50-60c624de73cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:53.692063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:53.743940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:53.745730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.746011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.746285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/85fb6b7fa0254f6694ac9f72062c761b/roles/60751164de814fd294ca92d31ff8170e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.751325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, user_id=85fb6b7fa0254f6694ac9f72062c761b, role_id=60751164de814fd294ca92d31ff8170e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:53.752775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:53.779730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 142/288] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:00:53 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/users/85fb6b7fa0254f6694ac9f72062c761b/roles/60751164de814fd294ca92d31ff8170e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:53.803737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571d97-2d6d-4e98-9bd3-3bab9d3b78de None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:53.843370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:53.844013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:53.844285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:53.844556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:53.857142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:53.862602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:53.864668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:53.865782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 147/289] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:00:53.873359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:53.950991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:54.071497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f639cc-8034-474e-81f8-ed034d51d51b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:54.125508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:54.126072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:54.126355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:54.126641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:54.127320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:54.127624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:54.129019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:54.141145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 143/290] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:54.145475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:54.301149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:54.301514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363de415-0c03-48e9-b7f2-4d2c5f4aea1a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:54.348941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:54.349606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:54.349884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:54.350159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] PATH_INFO: `/v3/users/64f7fed227e4422c9c801977c7b5bab0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:54.353939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] RBAC: Authorizing `identity:get_user(user_id=64f7fed227e4422c9c801977c7b5bab0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:54.355448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:54.357405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 148/291] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:54 2022] GET /identity/v3/users/64f7fed227e4422c9c801977c7b5bab0 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:54.363564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:54.470981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:54.484994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1718833-0915-4fd2-9002-82f05ba4ac57 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:54.518914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:54.519780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:54.520214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:54.520625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:54.612699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:54.614404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:54.732622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:54.747468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 144/292] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:54.753581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:54.901046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb2bf0-70e8-49e2-a2bc-cb5ef1be9467 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:54.946978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:54.947616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:54.947897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:54.948174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:54.949332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:54.950768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89cf0-6914-4e37-8ae3-ad7669cd7577 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:54.952703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 149/293] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:54 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:54.958578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:55.112815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab948e5d-84ed-4068-8da3-f891dce4c631 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:55.159044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:55.159666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.159947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.160219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.160944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=64f7fed227e4422c9c801977c7b5bab0) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:55.161275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:55.162709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a28c-0a2d-46a2-b6b4-d13078e7e1a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:55.172478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 145/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:54 2022] GET /identity/v3/role_assignments?user.id=64f7fed227e4422c9c801977c7b5bab0&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:55.320456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef18c-caac-4f90-85de-70b6a18cab6f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.320968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef18c-caac-4f90-85de-70b6a18cab6f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.321402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef18c-caac-4f90-85de-70b6a18cab6f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.323257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 150/295] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:55.329406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318540-a483-45ca-ae30-9365e6056d4c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.330004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318540-a483-45ca-ae30-9365e6056d4c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.330480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318540-a483-45ca-ae30-9365e6056d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.353487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68318540-a483-45ca-ae30-9365e6056d4c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:55.355005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68318540-a483-45ca-ae30-9365e6056d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:55.422711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:55.440005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 146/296] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:55.551176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5f24-c353-4410-9549-5ad5bd7125a8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.551526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5f24-c353-4410-9549-5ad5bd7125a8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.551717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca5f24-c353-4410-9549-5ad5bd7125a8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.553309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 151/297] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:55.559077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213b1ae-ec7c-4162-85cc-706fce9ae6f5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.559629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213b1ae-ec7c-4162-85cc-706fce9ae6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.560094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213b1ae-ec7c-4162-85cc-706fce9ae6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.582538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d213b1ae-ec7c-4162-85cc-706fce9ae6f5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:55.585571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d213b1ae-ec7c-4162-85cc-706fce9ae6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:55.677302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 147/298] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:55.732635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca5f24-c353-4410-9549-5ad5bd7125a8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:55.775498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:55.851450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca5f24-c353-4410-9549-5ad5bd7125a8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:55.868459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a48-8aac-40fc-bc96-90aa3226d4e1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.869426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a48-8aac-40fc-bc96-90aa3226d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.869893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a48-8aac-40fc-bc96-90aa3226d4e1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.872680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 148/299] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:00:55.876966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.877502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.877990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.896212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:55.896847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:55.897147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:55.897539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:55.902242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:55.903977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:55.909050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:55.914066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:55.914950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:55.916363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 152/300] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:55.920814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:55.986789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 149/301] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:55.987741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.033272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.040759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887c33b-ca37-41fd-887e-179da35858fc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.044138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.088743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.089479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.089823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.090163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.091410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:56.091794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.093542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.104129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a9924-733e-4db9-8dce-4b441365feac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.104962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 153/302] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:56.110120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.151285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.152067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.152409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.152800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.167527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:56.174043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:56.175133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:56.176141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:56.181463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 150/303] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:56.183169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.238274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66220c-bb83-417b-b308-9d999ef1ee45 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.274749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.275373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.275657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.275930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] PATH_INFO: `/v3/users/62b8a180bb024afba321ab24f443d4ea` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.277054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] RBAC: Authorizing `identity:get_user(user_id=62b8a180bb024afba321ab24f443d4ea)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.278535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a37dd-c183-4577-a1ee-93239adef20f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.280410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 154/304] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/users/62b8a180bb024afba321ab24f443d4ea => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:56.281642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4aa472-0c95-4477-a380-dd0533b5c241 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.281930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4aa472-0c95-4477-a380-dd0533b5c241 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.282214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4aa472-0c95-4477-a380-dd0533b5c241 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.285237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 155/305] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:56.286358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4aa472-0c95-4477-a380-dd0533b5c241 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.291983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0984897-a266-4e4f-88ee-3bc1f5ada841 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.299789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.314658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.328225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.328839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.329141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.329425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.330082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:56.330383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.331771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.339744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 151/306] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:56.341962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.392056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4aa472-0c95-4477-a380-dd0533b5c241 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.423727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.428029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.428029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.428029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.428029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.428029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb622a4-874c-4c95-b655-a74ea78dd8f7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.429519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 156/307] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:56.430669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.430975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.431225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.443393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1954-4e4b-4c1c-9d78-2fbf6e20b8e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.448952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:56.450174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:56.476572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.477246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.477524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.477795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] PATH_INFO: `/v3/system/users/62b8a180bb024afba321ab24f443d4ea/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.479183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62b8a180bb024afba321ab24f443d4ea, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.480595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.489220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 152/308] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Sep 1 07:00:56 2022] PUT /identity/v3/system/users/62b8a180bb024afba321ab24f443d4ea/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:00:56.489975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.523882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 157/309] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:56.524789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.526391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.554513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.629075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e1e1-a23b-4142-aa6a-953ed5a9384a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.641550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782367b9-2026-4754-be8d-128681b1d3e1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.677972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.679423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.679888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.680357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] PATH_INFO: `/v3/users/56df12c8413e4b00a8f6dbc6d96d9360` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.691581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.692198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.692479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.692750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] PATH_INFO: `/v3/groups` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.693417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.696308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.701217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] RBAC: Authorizing `identity:get_user(user_id=56df12c8413e4b00a8f6dbc6d96d9360)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.702764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.704842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 153/310] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/users/56df12c8413e4b00a8f6dbc6d96d9360 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:56.711730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.716176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 158/311] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Sep 1 07:00:56 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:56.824338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:56.827187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.838341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ee4f-bf41-471c-a7cc-28f3356cead9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.877044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.878448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.878905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.879452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:56.880374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, user.id=56df12c8413e4b00a8f6dbc6d96d9360, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:56.880857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:56.882570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:56.888068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 154/312] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/role_assignments?user.id=56df12c8413e4b00a8f6dbc6d96d9360&scope.system=all&role.id=60751164de814fd294ca92d31ff8170e => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:56.933575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:56.951326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdad12-b934-4fc7-81fb-1ea0de85495e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:56.996735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:56.997424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:56.997705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:56.997979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.010435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:57.016131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:57.017035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:57.017874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 159/313] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:57.023767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:57.131150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0668c-3399-4e65-a11b-3d4309312e35 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:57.170924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:57.171532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:57.171808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:57.172080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.172742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:57.173044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:57.174448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:57.183450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 155/314] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:57.189133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:57.293995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5daffba-6dbe-4d97-a8be-70956ac69515 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:57.328350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:57.329075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:57.329381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:57.329699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] PATH_INFO: `/v3/users/85fb6b7fa0254f6694ac9f72062c761b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.330831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] RBAC: Authorizing `identity:get_user(user_id=85fb6b7fa0254f6694ac9f72062c761b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:57.332272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:57.334203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 160/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:57 2022] GET /identity/v3/users/85fb6b7fa0254f6694ac9f72062c761b => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:57.340304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:57.360812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:57.450929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adfaf1-151b-49c1-9af0-2ff5818fa890 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:57.490852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:57.491490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:57.491771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:57.492044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.563534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:57.564903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:57.583258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:57.677856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 156/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:00:57.683585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:57.700017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:57.797301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad655331-11a3-4a99-bfd3-5a2b28d77d25 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:57.833400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:57.833950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:57.834230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:57.834505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.835651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:57.837173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bad93c-b079-4640-add3-37d0df7f3a38 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:57.839096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 161/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:00:57 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:57.844546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:57.947518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d46f4-58bc-4c06-97dc-dfdb8f7ff9d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:57.967879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:57.988852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:57.989491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:57.989768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:57.990040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:57.990770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, scope.project.id=daa88045f031406fb27a9cd2815c34c3, user.id=85fb6b7fa0254f6694ac9f72062c761b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:57.991074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:57.993059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baecd77-2d97-47af-9a37-5c4c155e7f78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:57.999744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 157/318] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:00:57 2022] GET /identity/v3/role_assignments?user.id=85fb6b7fa0254f6694ac9f72062c761b&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=60751164de814fd294ca92d31ff8170e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:58.527527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc11f-ca44-4c15-8219-fb32a4597318 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.528143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc11f-ca44-4c15-8219-fb32a4597318 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.528143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cc11f-ca44-4c15-8219-fb32a4597318 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.529751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 162/319] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:58.534658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce6c1-3218-4856-817f-bd5401aa950e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.535013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce6c1-3218-4856-817f-bd5401aa950e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.535357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce6c1-3218-4856-817f-bd5401aa950e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.553965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-be8ce6c1-3218-4856-817f-bd5401aa950e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:58.555149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-be8ce6c1-3218-4856-817f-bd5401aa950e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:58.623222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 158/320] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:58.807284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19789-65fb-4281-9e25-24a2cbe97c2e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.807581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19789-65fb-4281-9e25-24a2cbe97c2e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.807752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a19789-65fb-4281-9e25-24a2cbe97c2e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.809146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 163/321] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:58.814094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.814385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.814657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.833030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:58.834220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:58.843898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc1570-8e7f-4f99-a305-57980f9af2fa None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.844676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc1570-8e7f-4f99-a305-57980f9af2fa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.844961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc1570-8e7f-4f99-a305-57980f9af2fa None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.846662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 164/322] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:00:58.851682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:58.852035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:58.852380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:58.871126 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:00:58.872300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:00:58.905349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 159/323] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:00:58.906197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:58.940743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 165/324] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:00:58.946913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.004820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496f20-b970-440b-b6c1-817f477416c8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.039104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.039735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.040013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.040287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.050227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:59.055090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:59.055967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:00:59.061628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 160/325] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:59.062567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.074880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.164450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6ebbc-d916-43e6-9e03-86f5e1c3d5d3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.171362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.201871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.202489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.202764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.203033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.203693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:59.203990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.205371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.213691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 166/326] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:59.214541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.271151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.274666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34f25c-8620-4b8d-a408-37679c86808a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.283125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.308338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.308964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.309273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.309544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.310655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.312163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.313289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a777d37-dbd0-4e75-9f99-df6ba358d381 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.313714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 161/327] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:00:59.314470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.349803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.350426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.350748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.351025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.360846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:00:59.366646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:00:59.367608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 167/328] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:00:59.369628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.415645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1bd3c-33ca-4414-9cc0-d8a385ca210b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.448874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.449534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.449813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.450075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] PATH_INFO: `/v3/users/62b8a180bb024afba321ab24f443d4ea` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.451180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] RBAC: Authorizing `identity:get_user(user_id=62b8a180bb024afba321ab24f443d4ea)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.452615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.454203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 162/329] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/users/62b8a180bb024afba321ab24f443d4ea => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:59.454951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.469554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8dd9-3230-4904-a278-60aecd80b60b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.489555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.499309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.504321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.504918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.505226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.505498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] PATH_INFO: `/v3/groups` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.506119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.507606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.521464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 168/330] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Sep 1 07:00:59 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:00:59.522258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.558498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69320e-c900-489c-b273-c11e7a8098da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.595538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.596178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.596823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.597548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:59.597848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.599266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.608853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 163/331] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:59.614763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.644614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c27e09-d407-4cad-8c38-1715667582f8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.684172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.684789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.685067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.686091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.686091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] RBAC: Adding query filter params (role.id=55a4c413bf2645148a37872a21fe8eeb, user.id=62b8a180bb024afba321ab24f443d4ea, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:00:59.686418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.687878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.702173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 169/332] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/role_assignments?user.id=62b8a180bb024afba321ab24f443d4ea&scope.system=all&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:59.737347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.742972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d53aa-8aa2-4137-a9f7-d79a425b1399 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.793296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.793963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.794241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.794513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] PATH_INFO: `/v3/users/56df12c8413e4b00a8f6dbc6d96d9360` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.796059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] RBAC: Authorizing `identity:get_user(user_id=56df12c8413e4b00a8f6dbc6d96d9360)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.797893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55775a0-6751-4a68-9654-8cd37c0e8bff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.799374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 164/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:00:59 2022] GET /identity/v3/users/56df12c8413e4b00a8f6dbc6d96d9360 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:00:59.806188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:00:59.865290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:00:59.926029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1557d-ef5d-4654-bd3f-d3dcfc32372f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:00:59.964336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:00:59.964999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:00:59.965317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:00:59.965598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] PATH_INFO: `/v3/system/users/56df12c8413e4b00a8f6dbc6d96d9360/roles/60751164de814fd294ca92d31ff8170e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:00:59.967031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=56df12c8413e4b00a8f6dbc6d96d9360, role_id=60751164de814fd294ca92d31ff8170e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:00:59.968416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cf9b3-53d7-42eb-be03-4a2b20044cf4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:00:59.978851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 170/334] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Sep 1 07:00:59 2022] PUT /identity/v3/system/users/56df12c8413e4b00a8f6dbc6d96d9360/roles/60751164de814fd294ca92d31ff8170e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:01.668108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97345bb-c851-40ff-a5ec-bd162b786df2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:01.669345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97345bb-c851-40ff-a5ec-bd162b786df2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:01.669345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97345bb-c851-40ff-a5ec-bd162b786df2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:01.670496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 165/335] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:01.675644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc95b-46d3-4d30-81db-0691a28b0593 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:01.676103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc95b-46d3-4d30-81db-0691a28b0593 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:01.676179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6dc95b-46d3-4d30-81db-0691a28b0593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:01.696819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4c6dc95b-46d3-4d30-81db-0691a28b0593 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:01.698037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4c6dc95b-46d3-4d30-81db-0691a28b0593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:01.801060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 171/336] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:01.896670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681609c-e803-45ad-be3e-78db1f32c984 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:01.896986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681609c-e803-45ad-be3e-78db1f32c984 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:01.897257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681609c-e803-45ad-be3e-78db1f32c984 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:01.899229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 166/337] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:01.904854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f53a2-5262-4c0f-ac9d-8cc74b5531d3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:01.905160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f53a2-5262-4c0f-ac9d-8cc74b5531d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:01.905440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f53a2-5262-4c0f-ac9d-8cc74b5531d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:01.927721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c12f53a2-5262-4c0f-ac9d-8cc74b5531d3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:01.928924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c12f53a2-5262-4c0f-ac9d-8cc74b5531d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:01.981507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:02.019699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 172/338] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:02.201284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681609c-e803-45ad-be3e-78db1f32c984 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:02.318753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681609c-e803-45ad-be3e-78db1f32c984 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:02.359786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:02.360788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:02.361305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:02.361768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:02.373287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e13a-8a47-45fd-8063-c86feb47dace None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:02.373571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e13a-8a47-45fd-8063-c86feb47dace None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:02.373908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5e13a-8a47-45fd-8063-c86feb47dace None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:02.375531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 173/339] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:02.382326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:02.383099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:02.383388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:02.383670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:02.389408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:02.390259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:02.391046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:02.393693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 167/340] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:02.400522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:02.420201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:02.421436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:02.471395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:02.535771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 174/341] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:02.623901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:02.644105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58db-4156-47b2-9fdc-4fa359650a4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:02.711974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:02.713432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:02.713918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:02.714389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:02.715262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:02.715781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:02.717438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:02.733141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 168/342] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:02.739603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:02.829342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424fcf-a2e7-4a35-bc19-856b0a3909e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:02.838242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:02.889772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:02.890761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:02.891364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:02.891841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:02.912941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:02.918184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:02.919150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:02.920110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:02.923795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 175/343] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:02.930060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:02.963270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:02.987080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c634a0-0710-4205-8ce3-fcc06a9c95b2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.033432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.034287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.034707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.035120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.091698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1341e-58bd-4100-a5ce-3da0935ca96e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.127230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.129030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad840de4-cee7-44e6-87be-49ae89fcec99 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.139128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.139789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.140060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.140339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.141013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:03.141345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.143201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.154895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 176/344] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:03.156007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.191948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.240656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.261276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 169/345] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:03.262540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb949-2a75-4659-a4a2-2bff197f700d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.262824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb949-2a75-4659-a4a2-2bff197f700d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.263099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb949-2a75-4659-a4a2-2bff197f700d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.264447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 170/346] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:03.269791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb949-2a75-4659-a4a2-2bff197f700d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.271328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6aff1-2e7b-4518-9f91-90bdd6acc07b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.310266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.311274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.311746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.312220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.330511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b19db220-e6e4-4257-82ce-e5c763f9a722 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:03.336300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:03.337297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:03.338149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:03.342029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 177/347] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:03.343593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.344075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.344538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.369463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:03.370993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:03.383282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb949-2a75-4659-a4a2-2bff197f700d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.429447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.430086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.430742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.431021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.454763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 178/348] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:03.456121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.478719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.509633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.522124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.524234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060a978-e959-4d5e-9293-501feb9bb5a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.581753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df88b3d-4e57-4eef-8658-30632056a7a7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.621665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.622650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.623177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.623655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.625017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.626694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.628504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 179/349] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:03.629554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.664744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 171/350] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:03.665870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc3b3-beea-4fdb-8a23-f27c6aa98bfb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.666151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc3b3-beea-4fdb-8a23-f27c6aa98bfb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.666427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fc3b3-beea-4fdb-8a23-f27c6aa98bfb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.667939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 172/351] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:03.668693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fc3b3-beea-4fdb-8a23-f27c6aa98bfb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.766492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.793152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c37c-3e36-45cb-9c21-7f82fd723ed8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.815197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:03.832211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fc3b3-beea-4fdb-8a23-f27c6aa98bfb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:03.834611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.835739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.836019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.836287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.836955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:03.837295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.840089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cba7f7-c03b-4f4d-932d-3d091d29a7d1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.850098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 180/352] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:03.852024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.852309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.852587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.873140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:03.874328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:03.877968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:03.878611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:03.878887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:03.879156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:03.886190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:03.887662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:03.889539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 173/353] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:03.890316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:03.958403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 181/354] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:03.959678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.019403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4346e-7be2-4b1c-986d-ddca641528f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.050520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.068556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.070339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.071067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.071928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.073553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.076007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.077608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70632a4-6acc-47c9-9677-8a1db9dd9133 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.078046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 174/355] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:04.079030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.106271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.125792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.126496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.126779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.127054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.142234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:04.148111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:04.148995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:04.149892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 182/356] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:04.150593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.199767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbaa19-4f2d-4b44-bae4-10db84d30d5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.246327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.247096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.247708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.247708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.282676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1baf1-ed46-4515-80e6-d73998d82f7e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.324668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.325313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.325594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.325865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.326595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=1750242301a84f42b1c4b15f4e538db9) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:04.326899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.328332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.329717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.331356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea690b-8115-4265-a442-97679c6bf1b6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.332498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.337373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 183/357] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:04.338346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.357167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.467033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 175/358] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:04.468157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923f1bf-6cd1-483f-95cd-98a3ba5c99d4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.468441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923f1bf-6cd1-483f-95cd-98a3ba5c99d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.468715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923f1bf-6cd1-483f-95cd-98a3ba5c99d4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.470120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 176/359] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:04.470694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923f1bf-6cd1-483f-95cd-98a3ba5c99d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.480279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efaaa7-46f1-4dc3-a157-01074d22629d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.531041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.532209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.534594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.534594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.536484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.537787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.540045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 184/360] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:04.542007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.599111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923f1bf-6cd1-483f-95cd-98a3ba5c99d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.626614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.630209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.648798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.649491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.649768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.650032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.650739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:04.651037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.652416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.667194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 177/361] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:04.669366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.671133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b202a-927a-4058-8899-16ea8cbeb7ff None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.721326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.722000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.722330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.722604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.740429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-aa77ad64-3dac-4c0f-9ffa-305ba5fc4181 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:04.750417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:04.751760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:04.752576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:04.755214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 185/362] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:04.757017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.757474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.757752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.783663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:04.785278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:04.814118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a523a1-8899-4761-b613-e4633c8c57dd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.861541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:04.862196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:04.862459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:04.862737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:04.864570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:04.866109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:04.868520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 178/363] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:04.870172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.877714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 186/364] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:04.879066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:04.922203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.943020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:04.980615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb026f-6e13-4faf-9253-fae33fc9ccfc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:04.990265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c74771-9e08-4ef9-bb41-54133592a184 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.016937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.017910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.018381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.018870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.019783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=1750242301a84f42b1c4b15f4e538db9) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:05.020280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.022288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.028817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] PATH_INFO: `/v3/users/56df12c8413e4b00a8f6dbc6d96d9360` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] RBAC: Authorizing `identity:get_user(user_id=56df12c8413e4b00a8f6dbc6d96d9360)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.034150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 179/365] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.034604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 187/366] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:01:04 2022] GET /identity/v3/users/56df12c8413e4b00a8f6dbc6d96d9360 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.035441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.035732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.158454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e316-cdde-41a3-8cac-0d484018fc31 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.168495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d881a-6cdb-442c-83df-31d726259620 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.183172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.190258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.203649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.204664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.205235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.205725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.207100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.208395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.208964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.209084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.209405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.209681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.210900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:05.211207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.212574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.212677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 188/367] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.213666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.222093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 180/368] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.222948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.339078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be4632-edc5-46a5-8c18-d22151ca6296 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.343968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1daf6-f86f-4a6c-b843-f0020c41d168 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.388783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.388783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.388783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.388783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.389211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=daa88045f031406fb27a9cd2815c34c3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:05.389456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.390859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.397982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.398148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 181/369] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.398832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.399350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.399816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.401255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] RBAC: Adding query filter params (role.id=60751164de814fd294ca92d31ff8170e, user.id=56df12c8413e4b00a8f6dbc6d96d9360, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:05.401793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.404249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.418390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 189/370] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/role_assignments?user.id=56df12c8413e4b00a8f6dbc6d96d9360&scope.system=all&role.id=60751164de814fd294ca92d31ff8170e => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.419748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.450306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.521798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.530321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55c2b9-cb43-428b-a101-7d86f1a2de2d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.592922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.593504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.593786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.594059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.598705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320bb2c-0c46-4ba2-8c03-601426463e13 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.677912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.678637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.679402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.679678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.694423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.694948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:05.696315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:05.707008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:05.710086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:05.712089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 190/371] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:05.714629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.752302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.859349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 182/372] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:05.864839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:05.875934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed65879-ea83-4d5b-8b76-6e241ca09ac3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:05.911781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:05.944686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:05.945327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:05.945605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:05.945877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:05.946549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:05.946852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:05.948234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:05.969895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 191/373] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:05.971314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:06.068374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614eb63-6b57-481e-a98b-e9082eb0ab1b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.146838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:06.147962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.148254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.148513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.149755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:06.151169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:06.153139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 183/374] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:06.155423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaebe84-dc7d-43db-a764-6072d2415a7a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.162360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:06.182136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:06.229414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:06.230030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.230369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.230645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.310600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:06.333877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad8ce7-5857-4224-bf4c-5a233f392302 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.355955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:06.357471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:06.401871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:06.402425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.402702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.402974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.404619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:06.406082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:06.408053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 184/375] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:06 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:06.413669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:06.508944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:06.537492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e08f-33b1-4414-99bd-8c1ca8472ec8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.570504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 192/376] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 07:01:06.575894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:06.580895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:06.582389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.582669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.582941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.583677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=daa88045f031406fb27a9cd2815c34c3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:06.583982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:06.586026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62979-c69d-4d51-94ec-c7080c5e6e43 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:06.599348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 185/377] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:06 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:06.684433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:06.720438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f2f1-bf41-400f-b5b3-3724e10031b1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.772312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:06.772925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.773274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.773551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] PATH_INFO: `/v3/projects/b911f14b05394fe89de7092f0075d554` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.774675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] RBAC: Authorizing `identity:get_project(project_id=b911f14b05394fe89de7092f0075d554)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:06.776118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:06.779362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 193/378] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:06 2022] GET /identity/v3/projects/b911f14b05394fe89de7092f0075d554 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:06.784504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:06.784967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280adf-0fdf-4680-857d-b2f491e4a6f1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.785387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280adf-0fdf-4680-857d-b2f491e4a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.785585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55280adf-0fdf-4680-857d-b2f491e4a6f1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.787321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 186/379] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:06.804931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:06.805238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:06.805499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:06.842907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:06.844430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:06.939107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d53e9-8e63-445a-ac9f-20a6c77e1d40 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:06.974625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 187/380] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:07.014716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:07.015362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.015643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.015917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] PATH_INFO: `/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.025600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] RBAC: Authorizing `identity:get_group(group_id=714a9200f3214e7fb1a0ddd4d7e29650)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:07.027046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:07.028967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 194/381] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:06 2022] GET /identity/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:07.034757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:07.121513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:07.153256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0722a71-286b-4cf0-9617-413b7cd77802 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:07.191280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:07.191816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.192136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.192419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.193162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] RBAC: Adding query filter params (group.id=714a9200f3214e7fb1a0ddd4d7e29650, role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=b911f14b05394fe89de7092f0075d554) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:07.193477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:07.194943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:07.201191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 188/382] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:07 2022] GET /identity/v3/role_assignments?group.id=714a9200f3214e7fb1a0ddd4d7e29650&scope.project.id=b911f14b05394fe89de7092f0075d554&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:07.262441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:07.275245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:07.417589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134244b2-75dc-4787-9c15-b868ae930056 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:07.468638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:07.469211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.469494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.469765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.486692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:07.492114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:07.493033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:07.494151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 195/383] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:07.499597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:07.641039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c99e96-5e1a-49e2-8cf1-17dc16c6558b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:07.677143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:07.677758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.678038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.678310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.679021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:07.679322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:07.680707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40908bc-c325-4c0b-9413-0990fefc6c7f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:07.682123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:07.691484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 189/384] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:07.696789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:07.784296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:07.814518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7758f8c-2c4b-4331-acc6-1fed6daf1674 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.814823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7758f8c-2c4b-4331-acc6-1fed6daf1674 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.815066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7758f8c-2c4b-4331-acc6-1fed6daf1674 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.817162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 190/385] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:07.826318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21b2a-4dfd-4a3f-9ecd-8fec259f587f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.826600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21b2a-4dfd-4a3f-9ecd-8fec259f587f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.826874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d21b2a-4dfd-4a3f-9ecd-8fec259f587f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.830712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312190e-fd8a-42f9-a84c-47a80e00984a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:07.851767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e7d21b2a-4dfd-4a3f-9ecd-8fec259f587f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:07.853318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e7d21b2a-4dfd-4a3f-9ecd-8fec259f587f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:07.881380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:07.882616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.882896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.883167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.884323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:07.885957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e9590-8401-4e15-9a0d-7ad9978c85bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:07.887550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 196/386] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:07 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:07.888704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb837a3b-56e5-49da-b8f1-14909d773b37 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.888982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb837a3b-56e5-49da-b8f1-14909d773b37 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.889304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb837a3b-56e5-49da-b8f1-14909d773b37 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.890746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 197/387] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:07.896280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb837a3b-56e5-49da-b8f1-14909d773b37 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:07.942612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 191/388] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:07.943992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:07.944911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:07.945407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:07.967653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:07.968936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:08.036777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb837a3b-56e5-49da-b8f1-14909d773b37 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.042786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 192/389] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:08.078223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.078858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.079138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.079412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.081779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:08.083232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:08.085845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 198/390] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:07 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:08.091688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.153342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:08.204194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db882cbd-c983-4162-8fdb-d636fbf8830b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.245991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.246677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.247051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.247329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/groups/c3b514179ad84c48bad334ed6fd5651c/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.249356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, group_id=c3b514179ad84c48bad334ed6fd5651c, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:08.250969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:08.267798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 193/391] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 07:01:08 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/groups/c3b514179ad84c48bad334ed6fd5651c/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:08.378827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.382468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:08.433705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.521642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1f6a7-de52-4d42-9bdb-655190c173e8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.567153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.567355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1798-645d-44a5-81fb-a9652af23936 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.567844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.568205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.568483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.581071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:08.586551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:08.587427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:08.588223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:08.589320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 199/392] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:08.595180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.611182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:08.666475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.667040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.667320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.667593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.668583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:08.706062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:08.711079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:08.713176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:08.715079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:08.716506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cb95-712b-4d11-8452-daac02c12467 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.716646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 194/393] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:08.720028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.821852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.822475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.822757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.823035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.823868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:08.824012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:08.825424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:08.848894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 200/394] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:08.855010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:08.868018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee2cc5-d2cf-4dde-8584-fcd1a5ac94c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:08.914627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:08.915781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:08.916212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:08.916834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:08.918083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:08.918534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:08.920530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:08.947453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 195/395] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:08.962032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.007573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171d4ec-5811-4ea6-94c2-ab011f68f450 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.033559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:09.044713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.045453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.045651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.046078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.047211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.048774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.051026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 201/396] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:09.056080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.082069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:09.196621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59507922-f2ff-4390-a9b6-de98eb1414f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.230772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb51403-425f-486b-8022-aea636634190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.231398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb51403-425f-486b-8022-aea636634190 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.231673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb51403-425f-486b-8022-aea636634190 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.231942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb51403-425f-486b-8022-aea636634190 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.233038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb51403-425f-486b-8022-aea636634190 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.233751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa34bb-c24a-45a9-8d08-edbb05e659a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.234494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb51403-425f-486b-8022-aea636634190 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.240557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 196/397] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:08 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:09.241803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f019a-10fd-4770-93fc-130961eec8fd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.242087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f019a-10fd-4770-93fc-130961eec8fd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.242363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f019a-10fd-4770-93fc-130961eec8fd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.248001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 197/398] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:09.248798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f019a-10fd-4770-93fc-130961eec8fd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.275568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.276445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.277109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.277426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.278816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.281183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fff41-e626-41b9-bc97-7cc76b26c6b6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.283985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 202/399] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:09 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:09.285275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.285656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.285929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.313684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:09.315139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:09.370996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f019a-10fd-4770-93fc-130961eec8fd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.376705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:09.379154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:09.394285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 203/400] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:09.395095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.426678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.427485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.427854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.428216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.429918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.431444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.433391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 198/401] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:09 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:09.441433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.550582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f57b2-0ce6-4a8e-a221-36db65583928 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.602071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c2524-b41f-46db-880c-56e619726b2f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.617483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.618152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.618683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.618955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/groups/c3b514179ad84c48bad334ed6fd5651c/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.621668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, group_id=c3b514179ad84c48bad334ed6fd5651c, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.623114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.644205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 204/402] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 07:01:09 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/groups/c3b514179ad84c48bad334ed6fd5651c/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:09.658367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.659053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.659329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.659602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9/groups/c3b514179ad84c48bad334ed6fd5651c/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.662024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] RBAC: Authorizing `identity:create_grant(project_id=1750242301a84f42b1c4b15f4e538db9, group_id=c3b514179ad84c48bad334ed6fd5651c, role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:09.663580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:09.686752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 199/403] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 07:01:09 2022] PUT /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9/groups/c3b514179ad84c48bad334ed6fd5651c/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:01:09.770724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.840504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:09.906135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765691ad-cd57-4129-bbc3-5337706c1dd9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:09.942298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:09.942929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.943204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.943475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:09.955431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-559e2ba5-b876-42ba-9b74-bc3072cdfd4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:09.960857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:09.961780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:09.962570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:09.966758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 205/404] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:09.968101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:09.998487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187c1f-8526-4714-90e7-f8de37d5cb60 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:09.998788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187c1f-8526-4714-90e7-f8de37d5cb60 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:09.999025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187c1f-8526-4714-90e7-f8de37d5cb60 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.000483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 206/405] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:10.007668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.007943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.008212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.030550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:10.032414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:10.034429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:10.111092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aeb16-f2ef-456d-b337-c11aabdb08fb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.148188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 207/406] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:10.169542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.170166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.170443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.170709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.171469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:10.171766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:10.173228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:10.187470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 200/407] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:10.193269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.252200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:10.336980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ac03b-0dce-4f10-b1a4-500af5fbf29c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.383229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.383784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.384063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.384335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.385514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:10.386964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:10.388858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 208/408] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:10 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:10.394224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.513849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.537514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:10.544147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac7da-6d5f-4590-8574-5b7ccc3c3efa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.599979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.600818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.601097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.601391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.602593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:10.604042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:10.605843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 201/409] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:10 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:10.611747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.633394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ab093-1743-4216-ba50-1ffd4f625c3d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.646975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:10.684827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.685487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.685765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.686038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.697099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-87a79d61-ecb2-4672-924a-33424e1cc9d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:10.703465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:10.705609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 209/410] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:10.707545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e75e-165e-4a75-8f42-32b13fd1aecc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.709570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e75e-165e-4a75-8f42-32b13fd1aecc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.709570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e75e-165e-4a75-8f42-32b13fd1aecc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.709570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 210/411] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:10.713542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e75e-165e-4a75-8f42-32b13fd1aecc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.788547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b290a-3cd1-43ec-9468-22a6fd14f97f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.823361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e75e-165e-4a75-8f42-32b13fd1aecc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:10.834100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.835728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.836267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.836789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3/groups/c3b514179ad84c48bad334ed6fd5651c/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.840151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] RBAC: Authorizing `identity:create_grant(project_id=daa88045f031406fb27a9cd2815c34c3, group_id=c3b514179ad84c48bad334ed6fd5651c, role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:10.842464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064a01-1cd5-4d8b-8f1c-13a3110dde23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:10.859309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:10.859854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 202/412] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 07:01:10 2022] PUT /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3/groups/c3b514179ad84c48bad334ed6fd5651c/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:01:10.859961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.860189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.860467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.861268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:10.861570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:10.862340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:10.862857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:10.863060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:10.863542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:10.873805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 211/413] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:10.879228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:10.890698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:10.893350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:10.913622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:10.922276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:11.001481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 203/414] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:11.013638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1efb8-4587-4116-9a73-1cc7482018bf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.065600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.066267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.066555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.066827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] PATH_INFO: `/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.067956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] RBAC: Authorizing `identity:get_group(group_id=714a9200f3214e7fb1a0ddd4d7e29650)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:11.069469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:11.071392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 212/415] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:10 2022] GET /identity/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:11.076937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:11.188764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88c831-ab49-4700-808c-0b28bfbcc6e7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.227629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.228607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.229106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.229609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] PATH_INFO: `/v3/projects/b911f14b05394fe89de7092f0075d554` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.230969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=b911f14b05394fe89de7092f0075d554)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:11.232625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:11.234807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 204/416] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:11 2022] GET /identity/v3/projects/b911f14b05394fe89de7092f0075d554 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:11.240099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:11.320146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:11.347523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:11.375883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:11.380212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30a58f-7e53-4f14-bb56-5c0addf3e92b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.437818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.438493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.438827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.439041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] PATH_INFO: `/v3/projects/b911f14b05394fe89de7092f0075d554/groups/714a9200f3214e7fb1a0ddd4d7e29650/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.442695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b911f14b05394fe89de7092f0075d554, group_id=714a9200f3214e7fb1a0ddd4d7e29650, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:11.444465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:11.469930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 213/417] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Sep 1 07:01:11 2022] PUT /identity/v3/projects/b911f14b05394fe89de7092f0075d554/groups/714a9200f3214e7fb1a0ddd4d7e29650/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:11.486322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220b3cd-ade2-4526-9ba8-20dfe44fa5a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.525585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.526232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.526510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.526783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.539326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:11.544769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:11.545778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:11.546577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 205/418] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:11.554308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:11.666777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db6137-04d3-4d6b-b6b3-8f3a07dfa4a8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.704139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.705070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.705605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.706102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.707324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:11.707879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:11.710451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:11.721816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 214/419] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:11.730452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:11.822973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:11.842190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c86644-b1ec-42c6-a302-2939855774fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:11.886836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:11.887465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:11.887745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:11.888023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:11.971357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:11.972766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e869a9-9e0b-40ab-9f36-b83d66204bd1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:12.086204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:12.114351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 206/420] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:12.121381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.145450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:12.160975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbaa81-683b-4a1e-80dd-384624cabbe7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.161306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbaa81-683b-4a1e-80dd-384624cabbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.161553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbaa81-683b-4a1e-80dd-384624cabbe7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.163245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 207/421] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:12.168502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.168799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.169051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.191691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:12.192875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:12.269463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6220c9-4a4b-46d5-8feb-a35c14926f3c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:12.310484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:12.311476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.311963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.312452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.313915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 208/422] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:12.314668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:12.316310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa49da-650b-476e-83c0-737e7d769065 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:12.318456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 215/423] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:12.326355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.334417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48441-16a9-4d91-89c3-dd77913b2364 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.335023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48441-16a9-4d91-89c3-dd77913b2364 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.335513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b48441-16a9-4d91-89c3-dd77913b2364 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.337828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 216/424] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:12.341998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.342285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.342561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.370077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:12.371282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:12.511371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 217/425] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:12.522027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c68d4-5f72-4ff3-b627-bfe7fb85dad6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:12.535308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:12.557705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:12.558329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.558600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.558919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.560048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:12.561514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:12.563316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 209/426] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:12.569155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.588085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:12.626348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.723795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8997d40-c313-426f-b3d2-5a62d6836aa6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:12.785223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14c7f-7f4b-4a97-a7c6-b9c63bfb26e1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:12.790925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:12.791541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.791819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.792089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.792818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=1750242301a84f42b1c4b15f4e538db9) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:12.793169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:12.794681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:12.808610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 218/427] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:12.838616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.861915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:12.862579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.862863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.863142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:12.902206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:12.907307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:12.908298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:12.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 210/428] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:12.916791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:12.935062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:12.958390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ecafc-3be8-4202-b8b6-afed1d9c89eb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:12.996764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:12.997433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:12.997708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:12.998093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.009879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:13.015328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:13.017308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:13.018440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 219/429] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:13.024490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.026392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.099791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c386-8971-4c82-8c21-066855f89bcd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.132829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a5774-4ba6-4150-a38c-2e83e01b4acc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.140558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.141192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.141477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.141748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.142528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:13.142832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.144226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.156854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 211/430] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:13.160376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.175055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.175948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.176412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.176862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.177735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:13.178204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.179784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.189387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 220/431] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:13.198032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.204562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.320298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92765b6-054c-403e-8815-29b9d8d82ac4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.322594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.338311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ac0b7-5832-4342-90cf-c621c6db1c51 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.361149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.362145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.362693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.363142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.374359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.374853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.375132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.375405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.455402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.456980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d486-2d68-4734-bf6d-84f7200c3310 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.463099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.464544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.470455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.582507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 212/432] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:13.584186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcaaf4-81bf-4ec4-9d71-b3a0bc5b1bed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.585250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcaaf4-81bf-4ec4-9d71-b3a0bc5b1bed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.585735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dcaaf4-81bf-4ec4-9d71-b3a0bc5b1bed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.587188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 213/433] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:13.592993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.592993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.592993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.601073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.609987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 221/434] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:13.610999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.615685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:13.617208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:13.695602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 214/435] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:13.696774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.723229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898d44f-50a4-458f-b9f9-af6a9f431b5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.728187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.768885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.769877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.770369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.770883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] PATH_INFO: `/v3/projects/1750242301a84f42b1c4b15f4e538db9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.772923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] RBAC: Authorizing `identity:get_project(project_id=1750242301a84f42b1c4b15f4e538db9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.774867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.776664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 222/436] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/projects/1750242301a84f42b1c4b15f4e538db9 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:13.786181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.837080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaca79-32f2-4cef-a01b-a0e0aa8aedf3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.867591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:13.873155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.873790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.874071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.874337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.875450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.876890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682ee8e-f424-443c-b296-e46482f81da4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.883506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 215/437] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:13.883506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981fe4-29ab-4329-bdac-371766323a0e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.883506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981fe4-29ab-4329-bdac-371766323a0e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.883506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14981fe4-29ab-4329-bdac-371766323a0e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.883506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 216/438] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:13.888120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14981fe4-29ab-4329-bdac-371766323a0e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:13.909088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce822983-d4e3-4831-9809-ef35ec7f699c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:13.945267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:13.945888 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.946165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.946441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.947628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:13.949041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de7a85-0b87-4c33-b914-f87b6d8a9063 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:13.955606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 223/439] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:13.955606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:13.955606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:13.955606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:13.972419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:13.973641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:13.999029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14981fe4-29ab-4329-bdac-371766323a0e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.008174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.033527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.034150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.034423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.034694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.035783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.037294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.038884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 217/440] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:13 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:14.039634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.041742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 224/441] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:14.042482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.106979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.143172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4210b-5098-4e65-b03e-1e8e30092a20 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.148744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3a82c-6ebc-4738-87a9-b2d8990a06e6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.179223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.179826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.180101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.180370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.181087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=1750242301a84f42b1c4b15f4e538db9) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:14.181430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.182846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.184095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.184881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.185186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.185460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.197313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 218/442] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=1750242301a84f42b1c4b15f4e538db9&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:14.197313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.205152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:14.210383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:14.211427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:14.212220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 225/443] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:14.217715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.261969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.329129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c864fab-3b9f-4a34-bd78-847e1041b5cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.359177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ff9ad-736b-4b9c-b6b9-ad414f0032c3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.362965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.369515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.370295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.370955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.371230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.371965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=55a4c413bf2645148a37872a21fe8eeb, scope.project.id=daa88045f031406fb27a9cd2815c34c3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:14.372273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.373786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.380264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 219/444] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=55a4c413bf2645148a37872a21fe8eeb => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:14.381312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.399418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.399963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.400253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.400532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.401255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:14.401565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.403002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.415186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 226/445] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:14.420748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.524512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41336ac1-be7b-4f09-b6ba-5ac660c3598d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.547442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.559337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2bcf-6bcc-4e8d-ad14-948845879f8f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.568877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.570771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.570771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.570771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.587324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:14.593074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:14.594041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:14.594835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:14.595686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 220/446] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:14.600950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.601590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.602386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.602818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.603095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.665378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.684580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.686197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.721098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbdf54-0848-4c9c-bf8d-638f711ce0af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.767148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.767148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.767148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.767148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.767584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:14.767584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.769242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.779894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 221/447] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:14.784672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.802278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.812202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 227/448] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 07:01:14.817797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.888726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f4f7b-088a-49eb-917a-45d075773ac5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.906093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:14.916823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2df6-4a88-4b83-9478-3b7b9cda6876 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:14.923103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.923707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.923994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.924262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.950493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:14.951371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:14.951655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:14.951933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] PATH_INFO: `/v3/projects/daa88045f031406fb27a9cd2815c34c3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:14.953042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] RBAC: Authorizing `identity:get_project(project_id=daa88045f031406fb27a9cd2815c34c3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.954515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:14.956131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 228/449] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/projects/daa88045f031406fb27a9cd2815c34c3 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:14.961669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:14.990855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:14.992826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.019972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:15.060492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b69640-ecd5-4f40-b3e9-f41932cfdec2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:15.090524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 222/450] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 07:01:15.093554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:15.094176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:15.094454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:15.094728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] PATH_INFO: `/v3/groups/c3b514179ad84c48bad334ed6fd5651c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:15.095829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] RBAC: Authorizing `identity:get_group(group_id=c3b514179ad84c48bad334ed6fd5651c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:15.096342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:15.097328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.099141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 229/451] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:14 2022] GET /identity/v3/groups/c3b514179ad84c48bad334ed6fd5651c => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:15.104988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:15.113969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:15.196960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7d030-cd6d-4ccd-9f56-4407340a0bc7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:15.204660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cfb99-25be-47fe-8e9d-3bc029087d8b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:15.230571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:15.232241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:15.232932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:15.233259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:15.233574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] PATH_INFO: `/v3/projects/b911f14b05394fe89de7092f0075d554` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:15.234754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] RBAC: Authorizing `identity:get_project(project_id=b911f14b05394fe89de7092f0075d554)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:15.236205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.237845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 223/452] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:01:15 2022] GET /identity/v3/projects/b911f14b05394fe89de7092f0075d554 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:15.238644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:15.239314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:15.239592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:15.239865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:15.240579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] RBAC: Adding query filter params (group.id=c3b514179ad84c48bad334ed6fd5651c, role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=daa88045f031406fb27a9cd2815c34c3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:15.240887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:15.242417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.243302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:15.251434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 230/453] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:15 2022] GET /identity/v3/role_assignments?group.id=c3b514179ad84c48bad334ed6fd5651c&scope.project.id=daa88045f031406fb27a9cd2815c34c3&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:15.331463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:15.347993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fb2f-4dd6-4c3e-be74-eae6e2a3953f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:15.382980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:15.383611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:15.383885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:15.384157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] PATH_INFO: `/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:15.385298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] RBAC: Authorizing `identity:get_group(group_id=714a9200f3214e7fb1a0ddd4d7e29650)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:15.386742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2217b7e-b49b-4b24-ad88-143aedfa8dd8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.388311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 224/454] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Sep 1 07:01:15 2022] GET /identity/v3/groups/714a9200f3214e7fb1a0ddd4d7e29650 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:15.394204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:15.494957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192f871-c544-426a-a3e5-517becbb1829 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:15.528465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:15.529060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:15.529363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:15.529636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:15.530350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] RBAC: Adding query filter params (group.id=714a9200f3214e7fb1a0ddd4d7e29650, role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=b911f14b05394fe89de7092f0075d554) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:15.530658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:15.532072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1466e-f82d-4b27-8d1c-b98a192dfe19 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:15.537910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 231/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Sep 1 07:01:15 2022] GET /identity/v3/role_assignments?group.id=714a9200f3214e7fb1a0ddd4d7e29650&scope.project.id=b911f14b05394fe89de7092f0075d554&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:17.873246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304385d-8834-42c9-a1e9-1d5969282092 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:17.873788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304385d-8834-42c9-a1e9-1d5969282092 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:17.873788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304385d-8834-42c9-a1e9-1d5969282092 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:17.875420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 225/456] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:17.881358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea90636-be5f-432d-9485-89054b2206a9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:17.882104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea90636-be5f-432d-9485-89054b2206a9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:17.882104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea90636-be5f-432d-9485-89054b2206a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:17.910437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dea90636-be5f-432d-9485-89054b2206a9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:17.911814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dea90636-be5f-432d-9485-89054b2206a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:17.988264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:18.008888 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 232/457] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:18.078140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8e4b0-257a-48a1-b1c4-ccb84adbcd2d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.078459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8e4b0-257a-48a1-b1c4-ccb84adbcd2d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.078687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8e4b0-257a-48a1-b1c4-ccb84adbcd2d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.080265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 226/458] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:18.086005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b62-0ae3-47db-964c-296c5dddccbf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.086289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b62-0ae3-47db-964c-296c5dddccbf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.086559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828b62-0ae3-47db-964c-296c5dddccbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.113801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-db828b62-0ae3-47db-964c-296c5dddccbf None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:18.115226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-db828b62-0ae3-47db-964c-296c5dddccbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:18.199408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 233/459] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:18.244334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef386b4-272b-4ace-a405-f7d03d368db1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.244675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef386b4-272b-4ace-a405-f7d03d368db1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.244885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef386b4-272b-4ace-a405-f7d03d368db1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.246241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 227/460] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:18.251616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.251907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.252182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.270604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:18.271768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:18.338608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 234/461] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:18.412497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef386b4-272b-4ace-a405-f7d03d368db1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.536868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef386b4-272b-4ace-a405-f7d03d368db1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:18.580466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:18.581093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.581386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.581651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.592419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:18.593839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:18.614436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:18.615405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:18.616816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:18.616816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:18.616816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:18.618193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.619459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 228/462] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:18.624797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.635412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:18.693462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:18.744334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf491c-16f4-45ec-8b2b-a381e135de11 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:18.748635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acafd-6b0a-4336-9c00-434607bb301e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:18.783271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:18.783890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.784158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.784423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.790231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:18.791383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.791654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.791930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.792570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:18.793361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:18.795079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36098b02-6c51-4e0f-b2cc-c527622f34e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:18.795529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:18.796961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:18.814307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:18.815315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:18.816850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:18.816850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:18.816850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:18.816850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 235/463] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:18.816850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.817497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 229/464] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:18.819081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f266-96ff-4e24-9ef8-19550be9f900 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.820146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f266-96ff-4e24-9ef8-19550be9f900 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.820737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f266-96ff-4e24-9ef8-19550be9f900 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.822299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 230/465] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:18.823258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f266-96ff-4e24-9ef8-19550be9f900 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.897814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:18.927009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:18.931665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a79ea-6837-46bc-9a61-3c8108dbed58 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:18.937699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f266-96ff-4e24-9ef8-19550be9f900 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:18.967674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:18.968291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.968570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.968840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.975864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:18.976972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:18.977295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:18.977569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:18.978596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:18.978902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:18.979983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:18.980451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:18.981425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:18.997790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 231/466] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:18.997790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4557ccaa-ad0a-450c-830d-b0d45dc5c8ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:18.998035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:18.999567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 236/467] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.000454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.016851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:19.018052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:19.084510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 237/468] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:19.085409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:19.124556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170e6a8-255d-4012-ad49-5d481b81099c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.152365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:19.162312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:19.163845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.164328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.164874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.165740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:19.167379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:19.193344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 232/469] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:18 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:19.194682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:19.195081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:19.219954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593e8f-bae1-455c-ae0b-41368f84f4be None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.260959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:19.261599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.261873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.262142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.262798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:19.264402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:19.293416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 238/470] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:19 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:19.322094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc67b2c-97cc-4551-a38d-6fca6e3c9a99 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.362649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:19.363333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.363613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.363882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.364604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:19.364908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:19.366894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622cbdc-be24-437a-bd0a-fb399d02bdfd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:19.388093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 233/471] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:19.394822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:19.467089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb47e32-4ffa-438b-be86-6336c1a29502 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.467270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb47e32-4ffa-438b-be86-6336c1a29502 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.467546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb47e32-4ffa-438b-be86-6336c1a29502 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.469289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 234/472] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:19.479302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.479632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.479850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.504316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:19.506003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:19.542531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90749f58-8a35-4750-a56c-a40372633dac None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.561344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:19.592061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:19.592668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.592942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.593233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.593890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:19.595448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:19.606249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:19.609862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 235/473] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:19.611044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:19.633105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 239/474] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:19 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:19.747021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3aab9-1795-4b94-b720-c2afcb5420e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.798958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:19.799687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:19.799962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:19.800228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:19.814849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:19.816334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:19.843420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:19.844412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:19.845739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:19.845739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:19.845739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:19.845739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 236/475] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:19.852187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:19.979127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e369e8-5990-4cc7-8583-baeb6090cfbb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:19.981734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:20.018974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:20.027605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:20.028229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:20.028491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:20.028760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:20.029433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:20.029731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:20.031205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:20.048829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 240/476] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:20.050374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:20.104655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24c27b-1eea-45f4-856c-6aedd7faab8b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:20.108753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:20.155700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:20.156322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:20.156599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:20.156870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:20.157644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:20.158959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:20.161736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68228f8-bd50-4370-9304-c4df88222d9c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:20.178915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 07:01:20.178915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 07:01:20.178915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [parameters: {'id': '6106c5dfc9004ba2b4eff1381b0990be', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 07:01:20.178915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 07:01:20.199827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:20.200802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:01:20.201241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:01:20.202127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 07:01:20.206822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 07:01:20.208015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:01:20.210980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:01:20.212986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': '6106c5dfc9004ba2b4eff1381b0990be', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:20.215235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 07:01:20.217325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.219764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 237/477] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 07:01:19 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:20.220135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:20.220135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:20.220135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:20.220135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:20.223450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:20.259601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 241/478] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:20 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:01:20.349752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:20.359311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98b3ec-bb09-48a7-b0aa-4676b8539388 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:20.414667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:20.416312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:20.416824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:20.417305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:20.437327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7fedf229-ad3a-45ff-a918-18c5da55a108 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:20.442423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:20.443508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 07:01:20.444514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:20.446456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 238/479] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:20.452566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:20.566264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fbdbf-ac0d-498b-a1b4-a5970d6d77b8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:20.576703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:20.605557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:20.606086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:20.606363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:20.606634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:20.607296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:20.607598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:20.608964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a18f7-8832-43c6-9a5d-e36ab81dfbe2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:20.617740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 242/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:21.547892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4687b-1771-4bc6-b3dc-6e9021c76e9a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:21.549020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4687b-1771-4bc6-b3dc-6e9021c76e9a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:21.550089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4687b-1771-4bc6-b3dc-6e9021c76e9a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:21.551928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 239/481] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:21.557441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bea3-3917-4df5-9cce-f7830d7af91f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:21.558279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bea3-3917-4df5-9cce-f7830d7af91f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:21.558745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23bea3-3917-4df5-9cce-f7830d7af91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:21.579924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cd23bea3-3917-4df5-9cce-f7830d7af91f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:21.581462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cd23bea3-3917-4df5-9cce-f7830d7af91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:21.677347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 243/482] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:21.713291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be139d86-fdaa-452a-b8d9-2848b1482c93 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:21.713638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be139d86-fdaa-452a-b8d9-2848b1482c93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:21.713900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be139d86-fdaa-452a-b8d9-2848b1482c93 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:21.715498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 240/483] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:21.720864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:21.721182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:21.721460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:21.740119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:21.741421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:21.825309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 244/484] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:22.125521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be139d86-fdaa-452a-b8d9-2848b1482c93 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.143279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.193478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.273602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be139d86-fdaa-452a-b8d9-2848b1482c93 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.312646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c80e41-a45c-4827-b8b7-a90528bfcdce None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.321760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.322396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.322674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.322947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.333801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2c4a8380-fdde-429f-b53a-f6d1dcbc0d38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:22.339014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:22.339913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:22.340701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 241/485] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:22.341897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7be-32b2-4361-97da-069e4011e9c2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.342114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7be-32b2-4361-97da-069e4011e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.342389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ba7be-32b2-4361-97da-069e4011e9c2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.343613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 242/486] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:22.346004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba7be-32b2-4361-97da-069e4011e9c2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.360057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.363014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.363913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.364333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.364788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.377913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-be8d6f3a-f28d-4ec1-9709-2cf84be4bad8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:22.383509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:22.384383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:22.385224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:22.390940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 245/487] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:22.390940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.390940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.390940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.413917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:22.415400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:22.461144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ba7be-32b2-4361-97da-069e4011e9c2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.489846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 246/488] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:22.491260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.495813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.496418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.496693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.496964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.497646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:22.497946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:22.499331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:22.507858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 243/489] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:22.517808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.519450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.598779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774469-d223-43eb-8352-d5668f8fa08b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.635465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.636399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.636896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.637402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.638532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:22.639028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:22.640611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:22.654657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 247/490] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:22.656257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.683986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.747287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9149c-b3d9-430c-8bd6-b8e1d3491728 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.747480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.776103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4b55-d135-471d-a304-93b63f85c7df None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.793099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.793763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.794040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.794309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.806668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:22.808159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:22.816436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.817195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.817476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.817751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:22.823488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:22.824391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:22.826257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:22.826257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:22.826257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:22.826257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 244/491] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:22.826257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.831692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:22.833198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-536e8ea9-ee6c-491d-9b56-1db08db01e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:22.848168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:22.849103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:22.849976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:22.849976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:22.849976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:22.849976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 248/492] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:22.850732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bd9d-61d5-458d-b6d0-8cbc85870483 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.851008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bd9d-61d5-458d-b6d0-8cbc85870483 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.851277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45bd9d-61d5-458d-b6d0-8cbc85870483 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.852698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 249/493] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:22.854647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45bd9d-61d5-458d-b6d0-8cbc85870483 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:22.940949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df76b96-38a5-4387-8000-36620c8602a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.965197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45bd9d-61d5-458d-b6d0-8cbc85870483 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:22.965550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:22.978299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:22.978934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:22.979264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:22.979537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:22.992495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:23.000259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:23.001374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:23.002350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:23.003090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.003090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.003090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.003090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.003379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.003379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.004216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3dae2-4a4b-4b35-9f78-c23d0337328e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.010050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 245/494] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:23.010521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.011474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.026615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 250/495] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.027899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.028220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.028482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.047011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:23.048092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:23.115401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa9bd-416d-4d7d-be99-cd79cad55020 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.118411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 251/496] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:23.119713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.169016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.169666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.169999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.170279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.170927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.171481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.173347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.193338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 246/497] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.195064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.232050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39076db8-1893-468b-a87d-c8c6430f46a8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.254132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.259143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.271465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.272263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.272557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.272893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.273671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.274037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.275560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.283875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 252/498] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.284286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.318086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12479818-a661-426a-bc81-9d79467bf63d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.352560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.353222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.353498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.353766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.361023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.362571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e07f3f56-3910-47ed-93f2-b3bbc9159920 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:23.376212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:23.377334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:23.378446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:23.382527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 247/499] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:23.383775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573686-6ff8-4ef0-a2c3-ab6ca815f765 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.384057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573686-6ff8-4ef0-a2c3-ab6ca815f765 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.384336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc573686-6ff8-4ef0-a2c3-ab6ca815f765 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.385675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 248/500] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:23.386375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc573686-6ff8-4ef0-a2c3-ab6ca815f765 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.398820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c5d21-0fde-4b16-a462-62d90aa5e3f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.432967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.433616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.433891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.434159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.441979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.443533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:23.456427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:23.457534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:23.458660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:23.462213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 253/501] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:23.463054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.486625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc573686-6ff8-4ef0-a2c3-ab6ca815f765 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.488119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.509175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.521389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.522013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.522294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.522566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.533619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.535060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:23.549702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:23.551450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:23.553082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:23.553082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:23.553082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:23.553082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 249/502] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:23.553082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.568176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc0c11-d970-44d2-8e03-1cf234b687e7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.600847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.601591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.601873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.602146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.602931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.603237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.604676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3767048-2c83-4dbb-bfc4-91078d14a11c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.640781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 254/503] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.642184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.642474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.642751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.655077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a57e5-5989-4207-b870-474f6ac2c2d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.660131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:23.661355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:23.688977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.689638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.689917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.690189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.700090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:23.705232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:23.706331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 250/504] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:23.707335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.723184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.732152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 255/505] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:23.732984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.776304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:23.818750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc329df-36ee-4b87-b3b2-bf4d5f44eede None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.848908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6f424-619d-4274-b3c5-3643a4fbcc8e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:23.853581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.854316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.854697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.855069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.855896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.856301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.857879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.884329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:23.886212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:23.886505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:23.886801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:23.887967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:23.888443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:23.890234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:23.907489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 251/506] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.907689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 256/507] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:23.907780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.907861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:23.984789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.004128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.011093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867342f6-7312-489a-a535-c42c466bc44f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.015373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238412c6-f347-4d43-a7d1-fc64c3dda1f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.044902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.045541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.045819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.046089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.046739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:24.047038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.048394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.049206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.049807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.050084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.050362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.056035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 257/508] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:24.056816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.116236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.117768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.155654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174ea36-f7c6-491e-83f0-ca64e439e8c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.185238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.188982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.189622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.189903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.190173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.215334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.218681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 252/509] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:24.219515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.259595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.260971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.316248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092012fe-5639-44b8-80a8-e2367fad7b29 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.348894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.349622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.349973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.350866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.356560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 258/510] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:24.357479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.357871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.359373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:24.371659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:24.372720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:24.373854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:24.377344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 253/511] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:24.377952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.407716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.415618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.458089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ceb60-2bc9-4ae6-9c3f-8e7717adca1a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.475015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bb7bb-3194-495a-8381-f1c1bfb8a845 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.491368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.491964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.492228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.492502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.502449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.503900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.510244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.510806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.511087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.511357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:24.518718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:24.519757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:24.520781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:24.520781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.520781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:24.520781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 259/512] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:24.520781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.523274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.524825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:24.539388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:24.540482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:24.541383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:24.541383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.541383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:24.541383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 254/513] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:24.541604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.621136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe31b9e-1e5a-492f-84ff-2a6fc4ed1e13 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.629735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.632346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.648758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf5cb8-72e9-4ebe-a034-4a2a97e65d74 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.655077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.655702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.655982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.656254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.667335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.668784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.681356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.681968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.682243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.682512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:24.682726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:24.683584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:24.684436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:24.684436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.684436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:24.684436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 260/514] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:24.684644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.693645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.695136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:24.708806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:24.710339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:24.716077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:24.716077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:24.716077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:24.716077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 255/515] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:24.716077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.791298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbea94-c57a-46fd-87da-20490b272955 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.815052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beb60e-f3b3-46d8-812a-86c1fede385a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.825245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.825858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.826118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.826390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.827108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:24.827413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.828848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.845860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 261/516] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:24.846667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.848467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.849072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.849381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.849657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.850903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:24.851208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.852567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.854931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.864791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:24.866152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 256/517] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:24.866934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:24.948209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54410ba-1919-4d09-ba11-5c1bafebd609 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.964312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d0e5-7b0f-4fdd-9a8d-9c73d9353e5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:24.981701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:24.982301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:24.982582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:24.982854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:24.983486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:24.983789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:24.985174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:24.996154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 262/518] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:24.996927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.002505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.003109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.003382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.003655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.004292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:25.004592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.005990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.017600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 257/519] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:25.018326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.072441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.081487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.103650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bcaee-87d8-45a8-a538-6739782444c1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.115628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414711-935c-4dbf-b3c8-72608dc16049 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.139343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.139948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.140221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.140494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.141158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:25.141454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.142857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.152081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.152692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.152972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.153271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.156512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 263/520] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:25.157339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.225595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.227061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.253230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a670e-b263-49e6-996d-5f6de4395179 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.282980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.286761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.287353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.287651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.287918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.288543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.289988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.299616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.312197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 264/521] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:25 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:25.312986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.329640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 258/522] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:25.330877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.433234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daa428-25c3-4e41-8e95-77fd437cfdcd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.438819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac551ed-f2e9-4b2b-85b1-e4f431a9e87e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.475340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.475970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.476244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.476515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.477571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.478201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.478475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.478747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.484606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.486185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.486402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.487794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:25.503436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:25.504341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:25.505202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 259/523] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:25.507553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:25.507666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:25.508787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 265/524] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:25.510268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.529665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.570231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.623109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184e02c-7c70-4fad-8baa-0b7c3c28c749 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.624093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17efad-8e57-47c3-a878-44679e2427ac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.658643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.659231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.659508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.659781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.661995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.662769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.663050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.663324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.666427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.667874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.674345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.675788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:25.679537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:25.680450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 260/525] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:25.686137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:25.696340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:25.697303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:25.698904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:25.698904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:25.698904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:25.698904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 266/526] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:25.698904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.778283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.800889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414279b-0903-4b02-92b9-5ceb844be5d8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.807131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee38802-22c7-44ae-b56d-4cde8b72e524 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.808676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:25.839341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.840157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.840444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.840777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.841530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:25.841901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.843271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.845236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:25.845853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:25.846134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:25.846410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:25.847394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:25.847499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:25.848888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:25.867907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 267/527] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:25.867907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.872839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 261/528] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:25.877324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:25.974637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a39ea7-28b2-415a-abac-9d23252b0f6a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:25.989509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62b2b-0280-4540-98c5-777d2da09426 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.016079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.016964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.017273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.017958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.018264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.019689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.025220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.025828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.026109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.026383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.027091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.027318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.028675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.030282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.037164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 268/529] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:26.038573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.047425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 262/530] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:26.047425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.071473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.146585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609e99-f1b1-4a7d-99bd-bfd1e48ce21e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.150559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7c33-6880-4f0e-9c89-a4fa56767f1d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.181556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.182156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.182437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.182709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.183468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.183739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.185226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.185822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.186428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.186833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.187092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.190873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 269/531] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:26.191678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.282824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.294785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.296290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.325157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.330808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab06507-bffc-4dbf-9f97-3b6c67e12674 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.371716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.372325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.372602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.372876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.373635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=9a9b0f324af44c0f8827b31befa6939b) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.373948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.375385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.382354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 270/532] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/role_assignments?user.id=9a9b0f324af44c0f8827b31befa6939b&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:26.383430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.424600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 263/533] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:26.432559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.549817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4c03d-99cc-4a92-bede-209a124ca4d9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.586201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.593255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecf1c5-8712-41ca-b6b7-afcb5eeffd53 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.613805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.614754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.615224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.615668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.632249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.632988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.633370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.633643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.634069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.635883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.644995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.646569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:26.659548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:26.660426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:26.661316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:26.669140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:26.670033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:26.670901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:26.670901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:26.670901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:26.670901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 264/534] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:26.671137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 271/535] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:26.671206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.671271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.701882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.849693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1797fca-6dfa-47a8-a473-343019f0727f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.870555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084bffc-559f-463d-83fd-50ea8420ef20 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:26.896614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.897283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.897562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.898342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.898990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.899569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.901626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.920857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 265/536] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:26.922429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:26.923040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:26.923320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:26.923593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:26.924288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:26.924595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:26.926147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:26.927536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:26.944544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:26.953209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 272/537] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:26.960674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:27.077463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:27.096801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c6b8-c110-4176-90bc-3a750f6f91c0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:27.136246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:27.136956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.137265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.137541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.150831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa319be-0ae5-4090-8815-c37b93fbabe9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:27.154294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:27.155775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:27.173914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:27.174900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:27.175748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 266/538] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:27.181442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:27.215917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:27.216467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.216747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.217020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.217789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=92e78fff3061449ba36aafbc1b3f7eaa) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:27.218097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:27.219528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:27.235649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 273/539] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:26 2022] GET /identity/v3/role_assignments?user.id=92e78fff3061449ba36aafbc1b3f7eaa&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:27.286690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:27.337470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266298e5-1731-4427-9c5a-c56bbba96869 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:27.376037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:27.376655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.376934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.377237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.377947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:27.378297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:27.379759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d0d2b-6a11-4241-b127-ecf7fdc89ca2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:27.399097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 267/540] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:27.405094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:27.484544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d4aa-63c8-4d4b-869c-f5cdba07ec45 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.484729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d4aa-63c8-4d4b-869c-f5cdba07ec45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.485017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d4aa-63c8-4d4b-869c-f5cdba07ec45 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.486664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 268/541] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:27.492197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.492485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.492763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.516581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:27.521143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:27.543969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f852c04-1159-4e08-b7b3-3c4cfbd25dd7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:27.607659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:27.608267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:27.608545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:27.608816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:27.609583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=605f796f654b4b98aa29355fd4109aae) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:27.609893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:27.611334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:27.624771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 274/542] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:27 2022] GET /identity/v3/role_assignments?user.id=605f796f654b4b98aa29355fd4109aae&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:27.627342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 269/543] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:27.968838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:27.973212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:28.080477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459639-1c49-4448-b740-c632ec063829 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:28.122287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:28.122919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.123196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.123466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.142019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:28.148157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:28.149047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:28.149875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 275/544] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:28.155326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:28.224890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:28.280878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac07de7-85f7-48e3-b416-1bdb7e8f66c0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:28.329336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:28.329946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.330222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.330546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.331150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:28.331448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:28.332824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2152a23-7e78-4ea6-bf2d-234de4401a7f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:28.345291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 270/545] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:28.347851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:28.421623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:28.465442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563893-6c16-4b11-93a6-c9c2c17968ab None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.465763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563893-6c16-4b11-93a6-c9c2c17968ab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.465993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563893-6c16-4b11-93a6-c9c2c17968ab None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.467317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 271/546] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:28.472387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf6e3-a311-4387-b2dc-95adaa74ee2a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.472661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf6e3-a311-4387-b2dc-95adaa74ee2a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.472931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaf6e3-a311-4387-b2dc-95adaa74ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.477462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914c258-bc9b-4d3a-9546-e7e78d5d07f3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:28.505828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5dfaf6e3-a311-4387-b2dc-95adaa74ee2a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:28.507629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5dfaf6e3-a311-4387-b2dc-95adaa74ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:28.543738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:28.544360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.544631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.544898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.564609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:28.566098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:28.592015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:28.592934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:28.593817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:28.593817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:28.593817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:28.597232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 276/547] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:28.599220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:28.620738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 272/548] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 07:01:28.711113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746470-0f25-45ce-985c-a6ba6502b582 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.711428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746470-0f25-45ce-985c-a6ba6502b582 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.711666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746470-0f25-45ce-985c-a6ba6502b582 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.716885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 273/549] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:28.718074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.718355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.718673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.726001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f7de-57c5-4225-bfca-2f85c5be5e0d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:28.753220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:28.757891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:28.759813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:28.796429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:28.797035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:28.797339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:28.797613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:28.798255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:28.798557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:28.801053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:28.828675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 277/550] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:28.833895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:28.857532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:28.910340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 274/551] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:28.950985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099ddcf-0e57-4686-9d13-85816927001a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:28.953447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.003586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.004222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.004565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.004992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.016089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.017565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:29.042241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:29.043394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:29.044339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 278/552] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:29.054617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.106308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0f455-e3cf-42e1-9ba1-d4e7dd518754 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.144398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.145778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.146718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.147966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.148495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.160586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:29.167214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:29.168101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:29.169258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:29.172977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 275/553] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:29.179054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.228287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.281346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37af3-efca-4de6-ba04-8d7a09e66c6c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.300015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed18d1-b62d-4707-ae74-341c6b93708f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.318377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.319144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.319425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.319744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.320471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:29.320770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.322288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.337997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.338946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.339439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.339924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.340780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:29.341317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.342909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.346386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 279/554] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:29.347345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.352948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 276/555] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:29.358114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.454640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.467488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c9f41-05c1-4600-85f6-e06d6c801cd6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.473750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.479198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8652-aa5d-4bfe-8fb2-557e1ed1e897 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.511928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.513225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.514039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.514502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.521471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.522738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.523079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.523414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.527771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:29.533555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:29.534474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:29.535309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:29.536938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 280/556] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:29.541247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.542164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.542729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:29.560717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:29.562254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:29.563169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:29.563169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:29.563169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:29.565196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 277/557] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:29.566840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.679313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6107d2-c5fc-48a0-827c-efae02e630e2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.696479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf09a-c3eb-4e2b-a1ec-095bc68cbf38 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.721738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.722340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.722623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.722895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.738291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.738907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.739193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.739488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.740128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:29.740431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.741883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.745762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.750936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 278/558] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:29.752472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.769190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.806961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.808429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d01956-5628-45c6-bb0f-d748607388f9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.867326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695e993-da22-471c-a10e-17bd7b7b5b31 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:29.901149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:29.902218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.902495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.902762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.903396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:29.903692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:29.905064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:29.915279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 281/559] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:29.916822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 279/560] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:29.917209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883572a-431f-4bcc-b0af-fa4069fd61d3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:29.917658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883572a-431f-4bcc-b0af-fa4069fd61d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:29.918117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883572a-431f-4bcc-b0af-fa4069fd61d3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:29.923432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.923894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 282/561] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:29.924977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883572a-431f-4bcc-b0af-fa4069fd61d3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:29.985547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:29.998951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.027370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6fbeb-25b5-4058-9370-5a42cf72df9e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.037078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883572a-431f-4bcc-b0af-fa4069fd61d3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.064284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.064915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.065218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.065516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.073028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.073676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.073963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.074235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.075782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.077251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.084046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.085823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:30.091300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.092403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:30.093571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:30.093571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.093571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:30.093571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 280/562] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.093571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-27c28bb4-87c0-4d59-a3b1-ac6062ca9fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:30.102149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.108064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 283/563] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf7-a812-4edf-a33c-19c643f0b536 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf7-a812-4edf-a33c-19c643f0b536 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.109513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf7-a812-4edf-a33c-19c643f0b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.123177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-26b09bf7-a812-4edf-a33c-19c643f0b536 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:30.124354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-26b09bf7-a812-4edf-a33c-19c643f0b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:30.190212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 284/564] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:30.191481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45917c7-cfba-4f69-8ccb-026d3715d8e5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.191772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45917c7-cfba-4f69-8ccb-026d3715d8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.192046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45917c7-cfba-4f69-8ccb-026d3715d8e5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.193305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 285/565] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:30.194560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45917c7-cfba-4f69-8ccb-026d3715d8e5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.217524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532f54-fce8-44aa-8154-d7a410b74488 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.232534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.249542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.259140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.259982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.260329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.260675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.268488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.270005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:30.281032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.282183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:30.283177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 281/566] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.283792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.311180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45917c7-cfba-4f69-8ccb-026d3715d8e5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.344668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.345298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.345575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.345846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.346476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:30.346770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.348140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d86fc8-cc2a-4536-9c89-ffc88fd72f3f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.364232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 286/567] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:30.364232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.364232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.364232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.380316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:30.381550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:30.403971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ea47-a322-4615-b38d-f646d181d02f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.440089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.440691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.440966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.441275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.441909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:30.442338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.443562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.453154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 287/568] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:30.453154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.456611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 282/569] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:30.456611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.485346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.505547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.561509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200645e-e39e-4e5c-ab1d-b7a35a6cfd97 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.562184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e6db-6cf2-42dd-83bc-291545aa0ada None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.595819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.596425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.596707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.596983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.597736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:30.598034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.599483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.604414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.605039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.605352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.605628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.612965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.614774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.623691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 288/570] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:30.623691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:30.626845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.627774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 283/571] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.633739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.707178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.726811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f426182-d5d0-4c53-bbbc-bbe82fe67852 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.741996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946035a6-d762-4df9-9c92-94c0e00374de None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.748969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.768377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.769010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.769332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.769610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.776254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.776383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.776921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.777221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.777498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.777730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.787639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:30.788624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.789497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:30.790381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 289/572] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.790887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:30.792901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.793998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 284/573] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.795217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.893300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62b0b9-10c2-4472-b86b-1e4fa8f2c821 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.895621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6df4d8-a969-4e44-8762-3e017e83c667 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:30.925572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.926198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.926476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.926747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.927461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:30.927763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.928836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:30.929483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:30.929753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.929843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:30.930020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:30.940414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:30.941882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:30.943563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:30.946803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 285/574] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:30.951479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:30.957501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:30.958579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:30.964201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:30.964201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:30.964201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:30.964201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 290/575] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:30.964201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:30.974635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.049023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf35c6-e0bd-48a1-b24b-7f25cc24573a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.067730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52bbe-3b42-4119-acba-49ac16e40c29 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.082258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.082883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.083161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.083434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.093305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:31.098103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:31.099047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:31.099864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 286/576] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:31.101059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.101180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.101255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.101255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.101255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.101770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.102127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.103471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.115226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 291/577] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.116574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.171959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.185015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.202117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea0519-b4c2-42ae-8ba2-0bc3f684f926 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.216278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cae98-63c4-49c6-92ac-48266cef9b3b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.237272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.237906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.238185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.238460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.239119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.239421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.241682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.249901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 287/578] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.250686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.250790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.251459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.251739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.252007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.262348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.263771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:31.277788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:31.278788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:31.279668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:31.279668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.279668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:31.279668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 292/579] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:31.279668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.355021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13029c-f09e-47d1-8b3a-da6bc07b2b32 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.376204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1a903-413d-47be-b082-f4cbfe625f14 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.383003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.388744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.389376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.389652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.389922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.390555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.390853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.392218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.406496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 288/580] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.407277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.409640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.410238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.410510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.410870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.411589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.411891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.413921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.418684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.421959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 293/581] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.422735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.511788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d5f3b-21cd-48b1-ac48-3c60bc94bb39 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.518521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ef74-0de6-49b0-9891-19ec8beb0abd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.546255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.546846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.547119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.547386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.548091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=6834988c33e94b9cb41df95e4bb1bd88) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.548388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.549834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.552040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.553056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.553406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.553682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.555299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 289/582] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/role_assignments?user.id=6834988c33e94b9cb41df95e4bb1bd88&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.556071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.571176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.572680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:31.588528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:31.589788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:31.590890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:31.590890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.590890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:31.590890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 294/583] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:31.591192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.615267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.641456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.677728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e4549-167e-45e2-92b7-421516f8157c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.703229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b681d-2aac-402b-ad5e-4f6867267a15 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.725925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.726674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.727044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.727410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.728115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.728494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.730008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.738854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.739572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.739911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.740252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.746778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 290/584] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.746778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.752883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.754853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:31.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:31.768942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:31.770064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:31.773386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 295/585] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:31.773945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.858318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.864934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22885-d14c-4d50-8a9d-e077f55d0120 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.878861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9955d-647c-44fe-956d-bf4e08fd0b88 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:31.896525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:31.903812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.904451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.904723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.904991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.913178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:31.913780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:31.914063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:31.914334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:31.914973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:31.915276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.916619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:31.916727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.918088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:31.929194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 296/586] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:31.932727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:31.933690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:31.939321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:31.939321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:31.939321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:31.939321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 291/587] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:31.939564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:31.939637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.046532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded01c-d88f-40d7-b792-d6862febd4e4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.050373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05a9ab-77c2-427f-8bb5-088342f0127e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.083845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.084494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.084769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.085040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.088015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.088720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.089102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.089501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.090292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:32.090703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.091774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.092344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.093214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.095936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.105791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 292/588] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:32.106245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:32.107294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:32.108372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:32.109633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.114295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 297/589] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:32.115008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.156728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.216433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc18fa-ceb0-4794-bf75-538ef63d7fd5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.221926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30410b97-7477-40ff-a343-0993cd73c008 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.252111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.252740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.253023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.253327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.258777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.259470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.259842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.260208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.260347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.261017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:32.261455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.261835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.262989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:32.273788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:32.274944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:32.276090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:32.279515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 293/590] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:32.280194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.282754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 298/591] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:32.283806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.343311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.381957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f898a1-9a5d-4aa7-ad2a-54e76b66d9ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.383716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.388719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed31f8a-d6fa-472f-80fe-fc129a785d99 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.423354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.423931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.424212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.424485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.431329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.432022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.432393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.432765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.433581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:32.433980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.435486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.439693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=3d44b8c931e047408a2d9d4baa490806, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.441163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.447412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 299/592] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:32.448353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.459087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 294/593] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:32 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:32.460232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.580234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab55a5-924a-46c0-8a53-9d2d1f9d1479 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.599742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc777-3fe9-4b7d-b852-d786c6dc18e4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.625457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.626103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.626378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.626648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.632657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.637752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.639217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:32.652876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:32.654040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 295/594] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:32.655180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.660105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.660887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.661335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.661777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.662687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:32.663131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.664692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.678408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:32.688219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 300/595] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:32.690199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.798722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e847df-0b0f-4015-b218-2c2026cbaa79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.802670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d220f-9ea0-46a1-a78d-ce73e4d26a23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:32.835410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.836377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.836847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.837371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.838285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:32.838842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.840507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.846268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:32.847106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:32.847390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:32.847665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/9a9b0f324af44c0f8827b31befa6939b/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:32.863424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 301/596] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:32.864543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.865275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=9a9b0f324af44c0f8827b31befa6939b, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:32.867425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:32.889219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 296/597] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:32 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/9a9b0f324af44c0f8827b31befa6939b/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:32.890522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:32.927170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.024124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.062879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da81e47-e4fc-4f81-8fe1-6dca47f65c83 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.085177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b32d9-3d74-435e-92ce-a8fa947f8d4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.104615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.106004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.106476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.106936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.119673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.121446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.123977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.124859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.124859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.125166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.136265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.137793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:33.139101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:33.140008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:33.140875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:33.140875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:33.140875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:33.142962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 302/598] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:33.148063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:33.153307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:33.154237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:33.155106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:33.155106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:33.155106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:33.155106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 297/599] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:33.161610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.250586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.265082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d897d-0020-4688-9ae7-c70f8a92894e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.270197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c1555-3ca7-47ee-a9a5-4684c5cd2f33 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.284377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.305825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.306647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.307053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.307450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.308208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:33.308668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.310213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.313422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.314140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.314518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.314890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.315625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:33.316023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.317514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.325151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 303/600] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:33.330038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.339419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 298/601] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:33.344834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.479421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55226723-d1e2-4baa-a79f-40418fa2173a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.495536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486fbf0-14ce-4e44-a171-c17334d4e6a6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.544231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.544772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.545051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.545510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.546063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.546801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.547142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.547506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.553841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.555106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.555617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.556744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-77f0e007-f96d-47d0-8f2f-970e5e4e5efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:33.567866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:33.568987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:33.570069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 304/602] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:33.570689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782674f5-9e77-47a2-8ccb-f7b78c3f7332 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.570904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782674f5-9e77-47a2-8ccb-f7b78c3f7332 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.571178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782674f5-9e77-47a2-8ccb-f7b78c3f7332 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2fff5f5e-e766-4a7a-b112-f4d860f2dde9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:33.571976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:33.572820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:33.574150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:33.580435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 305/604] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:33.580435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782674f5-9e77-47a2-8ccb-f7b78c3f7332 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.580806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 299/604] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:33.582309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.582809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.583276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.589047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.605549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:33.607039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:33.636684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.683336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 300/605] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:33.684374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.734173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782674f5-9e77-47a2-8ccb-f7b78c3f7332 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.790716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9d287-f73c-4bd1-9171-59a8f235dfa9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.809021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.809591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.809867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.810137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.810831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:33.811132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.812662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.827346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:33.828157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:33.828580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:33.828954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:33.829875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:33.830283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:33.832125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:33.838112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 306/606] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:33.843463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.851251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 301/607] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:33.853467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:33.887121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.959890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:33.963570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fd825-db18-4b08-8966-9480660b5bd9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:33.995940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1e8b7-44fc-4ec8-96d2-d7160112dc00 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:34.002131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.003091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.003674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.004165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/605f796f654b4b98aa29355fd4109aae/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.021500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=605f796f654b4b98aa29355fd4109aae, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:34.023913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:34.039920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.040659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.041026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.041476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/92e78fff3061449ba36aafbc1b3f7eaa/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.059102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=92e78fff3061449ba36aafbc1b3f7eaa, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:34.060783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:34.062658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 302/608] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:33 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/605f796f654b4b98aa29355fd4109aae/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:34.086998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:34.091838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 307/609] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:33 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/92e78fff3061449ba36aafbc1b3f7eaa/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:01:34.188480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:34.223175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b4db5-ed6d-44ff-ad76-7c9799ebe8b0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:34.264709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.265381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.265723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.266000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.280136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:34.285283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:34.286333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:34.287421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 303/610] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:34.292839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:34.411364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bba2e-4951-4f31-b5e3-33e4d4edb78d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:34.436732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:34.459560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.460170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.460447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.460716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.461399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:34.461701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:34.463123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:34.471817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 308/611] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:34.477339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:34.624690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8738832-0812-4fdf-9996-175bdf74bf36 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:34.659928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.660895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.661442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.661892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.664030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:34.672740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:34.674417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8c9a994f-c3f0-4176-bca5-47ddc1899b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:34.689996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:34.690944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:34.691854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:34.691854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:34.691854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:34.691854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 304/612] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:34.698766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:34.847005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee990284-2e7e-491e-82b0-26f03d4195de None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:34.856431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ca9dc-0f04-4237-877c-d3dd41f2f308 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.856729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ca9dc-0f04-4237-877c-d3dd41f2f308 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.856979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ca9dc-0f04-4237-877c-d3dd41f2f308 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.859563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 305/613] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:34.863610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.863890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.864164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.890788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:34.891976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:34.909623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:34.910225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:34.910549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:34.910753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:34.911543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:34.911785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:34.913780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:34.937070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 309/614] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:34.941471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.011177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:35.029277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 306/615] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:35.103804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983cc44a-4038-4fb8-82d7-e402ba4c82e8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.151296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.151916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.152193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.152459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.170142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:35.171510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:35.186386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:35.187519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:35.188492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 310/616] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:35.193064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.314933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d27289-0802-49f9-9d3d-e5c668d934f1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.338733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:35.364147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.364761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.365031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.365329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.366094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:35.366398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:35.367837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:35.391942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 307/617] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:35.397550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.418953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.555775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:35.559220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c79f-7983-462f-bac4-80a9032aff78 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.579402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e60402-97d6-4618-9b52-f8083e1757c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.595862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.596792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.597308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.597876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.609151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:35.610806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:35.625340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.625963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.626289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.626566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ab90a349-e9a8-48f5-8fe0-210f68c1fb4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:35.627146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:35.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:35.628936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:35.628936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:35.628936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:35.635672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 311/618] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:35.635672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e7ab-6260-44ef-8a3b-93f7d7ef1b13 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.635672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e7ab-6260-44ef-8a3b-93f7d7ef1b13 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.635672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e7ab-6260-44ef-8a3b-93f7d7ef1b13 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.637680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 312/619] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:35.638624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e7ab-6260-44ef-8a3b-93f7d7ef1b13 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.645107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bbfe0979-23ef-41b4-aee0-c260c87c3139 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:35.650838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:35.651926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:35.652759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 308/620] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:35.653505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.654049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.654049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.681143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:35.682623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:35.717202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:35.766251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e7ab-6260-44ef-8a3b-93f7d7ef1b13 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.777551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 309/621] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:35.777551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.831519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.832509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.832989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.833483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.838542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:35.839185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:35.846839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:35.872094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 313/622] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:35.879504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:35.886783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685272-b5ad-4848-8f85-fe36eee35e00 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:35.908097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:35.959937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:35.961166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:35.962689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:35.963197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:35.964276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:35.965703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:35.968592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:35.984853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 310/623] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:35.992131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.052813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.073767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5b243-6f12-493c-9087-a599d9328d79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.132062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.133036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.133546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.134016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.141005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.142767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:36.154689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:36.155616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:36.156504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:36.158935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 314/624] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:36.164745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.187344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325491e-2719-4ff3-8c7a-e5785768a565 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.230541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.231812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.232282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.232974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.246206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.248188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.250164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:36.268648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:36.269676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:36.270562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:36.270562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:36.270562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:36.273217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 311/625] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:36.274428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.284959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0336040-1736-4f73-9cc6-f8a29df5d3ee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.324013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.324721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.325203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.325597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.337240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:36.343028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:36.344100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 315/626] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:36.349312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.374075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.397956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98340c8-d89e-4768-a900-89f27681498c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.434852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.435446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.435720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.435988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.436664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:36.436971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.438416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268aee0-bd13-433f-be4f-24f094a9827f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.448216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9056bc-317f-46c1-8550-94be2c7e6d31 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.450804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 312/627] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:36.452018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895b74-7283-49ac-a313-df6de9bcc2cc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.452302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895b74-7283-49ac-a313-df6de9bcc2cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.452570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10895b74-7283-49ac-a313-df6de9bcc2cc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.453778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 313/628] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:36.454460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10895b74-7283-49ac-a313-df6de9bcc2cc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.483002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.488668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.489311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.489595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.489873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.490516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:36.491369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.492739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88042fb3-c9dc-47a7-a68a-d27ecbcfc063 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.504580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 316/629] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:36.506083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e363-a24c-4ac1-adbe-902a711fcd53 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.506415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e363-a24c-4ac1-adbe-902a711fcd53 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.506742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e363-a24c-4ac1-adbe-902a711fcd53 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.508059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 317/630] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:36.508661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e363-a24c-4ac1-adbe-902a711fcd53 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.560526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10895b74-7283-49ac-a313-df6de9bcc2cc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.593278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.593874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.594177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.594450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.595102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:36.595403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.596779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826db89-c2dd-4946-a9b4-e5193767d2a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.602220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.610122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 314/631] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:36.610122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b78ea-f0b7-4c5c-9dc9-db1d80ef4f2f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.610122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b78ea-f0b7-4c5c-9dc9-db1d80ef4f2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.610122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b78ea-f0b7-4c5c-9dc9-db1d80ef4f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.611624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e363-a24c-4ac1-adbe-902a711fcd53 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.625759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-422b78ea-f0b7-4c5c-9dc9-db1d80ef4f2f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:36.626929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-422b78ea-f0b7-4c5c-9dc9-db1d80ef4f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:36.646096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.646743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.647652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.647929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/6834988c33e94b9cb41df95e4bb1bd88/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.662090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=6834988c33e94b9cb41df95e4bb1bd88, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.664056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.683630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 318/632] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:36 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/6834988c33e94b9cb41df95e4bb1bd88/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:36.683630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.704403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 315/633] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:36.705919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.706302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.706683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.730366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:36.732387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:36.737240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.833529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550a654-dafb-44e1-b429-fe111e4de803 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:36.839419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 316/634] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:36.839419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.884539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:36.885857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:36.886261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:36.886595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:36.896343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:36.899011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:36.908659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:36.917346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:36.918488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:36.920050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:36.921388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 319/635] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:36.925594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:36.988546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dae09-5cd9-4880-98c7-8fe465a63cc0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.035656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.036299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.036577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.036849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.047972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.049470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:37.066222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:37.067219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:37.068123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:37.068123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:37.068123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:37.072468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 317/636] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:37.073497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.074915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d498d31-6ce4-40ac-819f-b9fa8dec3a65 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.076984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.123753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.124957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.125452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.126180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.127587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:37.128335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.130631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.155919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 320/637] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:37.157024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.188516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9ff66-89fa-4d97-afbc-d5a3b024027a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.204462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.227348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.227987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.228267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.228543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.239001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:37.245375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:37.246315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 318/638] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:37.247248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.272717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dc48a-4545-4733-b8fd-cd488529c23b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.306543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.307430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.307716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.307986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.308629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:37.308933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.310322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.322379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 321/639] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:37.323696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.348561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.360781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5a8db-1566-4f46-a9a3-d24317caf597 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.397041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.398438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.398919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.399196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.427482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3414a-27d6-4c20-b972-edee983c486d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.442496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.462894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.463519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.463794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.464067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.470976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.472717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.473845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:37.479097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:37.480029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:37.480873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 322/640] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:37.481603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.567112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.584800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 319/641] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:37.585921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.595047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e55d26-f21e-4c3e-a437-a36d5b5ac1e9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.629559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.630160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.630438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.630708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.631933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:37.632238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.633639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.645793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 323/642] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:37.645793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.665047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.696147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43ec28-44b6-4ab0-8503-ee2eb228b447 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.733972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.734597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.734871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.735137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.743106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.744828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.750542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5c0c2-f6e9-414e-8819-a3aa7a0ffd91 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:37.759892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:37.760939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 320/643] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:37.761971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.798904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.800054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.800520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.800971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.802107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:37.803981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.805588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.821068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 324/644] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:37.821068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.824716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.878933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33900499-8403-4ce4-abf7-f2d379ef0881 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.907750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:37.913893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.914516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.914804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.915080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.925965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.927421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:37.935611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f96b7-d000-4161-a835-191ae520a593 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:37.942182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:37.943194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:37.944207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:37.944207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:37.944207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:37.944207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 321/645] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:37.944207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:37.978548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:37.979116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:37.979408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:37.979658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:37.991348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:37.992765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:38.007502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.008428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:38.009354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:38.009354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.009354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:38.009354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 325/646] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.009354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.059708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec21433-0eea-47dd-8170-613185c9d108 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.067792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.097733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.098342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.098623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.098897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.099630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:38.100452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.101945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.116810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfef72-9d1e-4ce9-849c-1ea334257c33 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.120604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 322/647] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:38.121688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.159692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.160325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.160614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.160874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.171326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.172773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.174190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:38.188328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.189348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:38.191023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:38.191023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.191023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:38.191023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 326/648] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.191023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.237025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1256d0c-6d92-4eac-959d-4ae8dc449105 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.274176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.274793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.275196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.275476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.276120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:38.276426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.277967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.290761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 323/649] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:38.291593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.306056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295c84-6d81-4654-859e-d14c6481f484 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.315434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.343065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.343682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.343961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.344236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.344883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:38.345236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.346579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.359368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 327/650] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:38.360202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.409082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393f8a4-971a-4c0d-b199-ad3fe6e512aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.410636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.445697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.446312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.446598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.446873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.470047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862367a-ee4e-4474-a1fd-7a6b7a724f2d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.504073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.504607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.504890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.505183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.505934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:38.506765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.508141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.524178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 328/651] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:38.524178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.528499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.530463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.549696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.628624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a039a6a-96f7-4690-99bc-5c0eecd7648d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.639355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 324/652] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:38.640205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.642467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.668362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.669087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.669468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.669818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.676704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.678755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:38.690136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.691207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 329/653] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.692594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.744938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e740d-7eef-4aa9-969c-b5e68b3497f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.779292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.779927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.780206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.780479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.787139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.788001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.789514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.800406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018be135-1a93-417a-9902-7bd95e70be08 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:38.801663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.802745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 325/654] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.804774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.837579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.838296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.838576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.838851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.846098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.847523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e90d030f-540d-4c6e-8938-d42e21cd1570 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:38.859129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.860210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:01:38.861368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:38.863065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 330/655] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.864256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe972c2-49bc-4047-b23a-1bd2ef75794d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.864539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe972c2-49bc-4047-b23a-1bd2ef75794d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.864813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe972c2-49bc-4047-b23a-1bd2ef75794d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.865366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:38.869463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 331/656] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:38.870172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe972c2-49bc-4047-b23a-1bd2ef75794d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.906024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65dff0-473a-4b3f-9ed1-b68b2cb46699 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:38.939499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:38.940116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:38.940391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:38.940660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:38.951055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:38.952483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:38.966952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:38.968109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:38.968999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:38.968999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:38.968999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:38.968999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 326/657] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:38.969337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:38.973703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe972c2-49bc-4047-b23a-1bd2ef75794d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.008136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.008732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.009014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.009320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.010014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:39.010325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.012092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.023674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 332/658] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:39.024669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.036417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.072352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aadd2-ed5c-4b80-ad87-a066470b4156 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.087701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.109677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.110278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.110552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.110819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.111540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:39.111843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.113288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5125c-de2e-49f9-b7e1-e9d2aef86d43 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.127269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f65a70-15cb-44b6-90f8-7f79bfc62b0e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.130103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 327/659] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Sep 1 07:01:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:39.131357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.131638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.131911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.149238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:39.150396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:39.161545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.162146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.162429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.163301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.164041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:39.164677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.165868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.184065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 333/660] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:39.184879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.217281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 328/661] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:39.218125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.256776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.291615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a631a0-c8bd-47ad-9876-65962a8a8ad6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.326135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc0c0-3991-4df0-92fa-79f7b6f63a3e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.326468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.327072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.327352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.327635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.328257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:39.328564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.329449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.329976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.342361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 334/662] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:39.343049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.369803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.370413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.370689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.370961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.371680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:39.371987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.373473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.379204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 329/663] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:39.379974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.489454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf1dab-9241-4305-a84b-d10b3daadf81 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.513100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad525d5-2720-40f9-adc0-3f62fbd1bd81 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.520788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.532654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.532654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.532923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.533224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.551250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.551787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.552064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.552333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.591121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.614588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.616571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.633874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.635582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.717521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 335/664] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:39.718590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.745273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.747692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 330/665] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:39.749097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.822958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:39.828240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425aae6-7947-4b58-be5a-614864639950 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.861316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f4a92-112a-4e19-af0a-0d4f18402d75 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:39.862725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.863347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.863621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.863895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.871478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:39.872916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:39.884851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:39.885813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:39.886722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:39.890665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 336/666] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:39.891275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.900293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:39.900931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:39.901239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:39.901516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:39.912186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:39.917031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:39.918062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:39.918899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 331/667] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:39.919598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:39.986578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.006404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d605da9-7f24-468c-914c-427fc99bc354 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.029378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b236df-1c89-40c8-adc4-67074dbc4534 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.049270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.050659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.051095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.051791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.057025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.066914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.067555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.067835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.068107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.072103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.073869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.078929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.080380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:40.090947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:40.092070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:40.093602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:40.093602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.093602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:40.093602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 337/668] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:40.097087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:40.100493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:40.101461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:40.102385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:40.102385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.102385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:40.102385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 332/669] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:40.102385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.203205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc2d60-85d4-4bc1-9441-5dc3c77a66e3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.211950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554fd83-f405-495c-a068-dba1f19bb00e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.240057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.241354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.241649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.241964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.242696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.242993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.244687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.250160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.250954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.251390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.251767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.252700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.253390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.253782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.255160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.261835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 338/670] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:40.262478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.265466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 333/671] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:40.266416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.310609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.370765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524c3d3-3b6c-4ab4-a662-3f93523c4f10 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.373556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f1a41-493a-4bb3-9461-9ccc65aa67b8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.406421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.407355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.407819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.408158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.408747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.408862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.409048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.409401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.409677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.409946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.410449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.410604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.410872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.412349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.422541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 339/672] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:40.426172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.428192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 334/673] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:40.429016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.494132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.527129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.530321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc451-c003-49ca-94db-530ad87ee813 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.544330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec52f5b-187a-4829-989d-c8aacda3f246 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.565216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.565843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.566125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.566400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.576304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.577855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.581997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.582599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.582882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.583156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.583880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=9a9b0f324af44c0f8827b31befa6939b) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.584190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.586163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.592187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 340/674] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/role_assignments?user.id=9a9b0f324af44c0f8827b31befa6939b&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:40.593178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:40.594064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:40.594946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:40.594946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:40.594946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:40.596354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.599099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 335/675] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:40.604480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.733428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76dac-3620-4829-b7f5-f102131e337d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.738167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cc360-6006-4867-a316-8688b8d7b9b5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.763312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.780949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.781379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.781754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.782019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.783266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.784126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.784586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.785032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.794481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.794648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.795950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.796054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:40.809529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:40.810529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:40.811502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:40.811960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:40.814720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:40.815715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:40.816160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 341/676] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:40.816160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.816290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 336/677] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:40.820320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:40.840360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:40.936211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840457d-68d3-4b8a-87cc-5ea63c376c0e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.965344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e0d5-2cb6-488f-8788-a37116a09112 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:40.972838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:40.973545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:40.973828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:40.974104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:40.974755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:40.975101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:40.976586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.001684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 342/678] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.003972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.012813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.013473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.013757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.014030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.014728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:41.015037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.016657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.037255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 337/679] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.038187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.059283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.121673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.136259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dd17a-63f2-4b19-b820-eb5ca48b71d9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.164442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c914a9-d4c9-4a6b-9246-f1370ada6571 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.173533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.174576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.174576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.174780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.175592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:41.176008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.177542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.192710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 343/680] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.194421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.211857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.212677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.213030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.213424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.221269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.222867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:41.235998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:41.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:41.237938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 338/681] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:41.238554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.328509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2dc1e-be0d-479e-9401-7bf613e0c301 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.330923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.350282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0be1b-5b7e-4b99-913b-0921003ec5ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.369273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.370126 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.370413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.370758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.371576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=605f796f654b4b98aa29355fd4109aae) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:41.371955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.373476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.385569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.385912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 344/682] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/role_assignments?user.id=605f796f654b4b98aa29355fd4109aae&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.387755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.391395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.392345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.392871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.393401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.394491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=92e78fff3061449ba36aafbc1b3f7eaa) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:41.394995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.396777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183439e-ce58-4960-85e1-cebc16f3378d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.404369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 339/683] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/role_assignments?user.id=92e78fff3061449ba36aafbc1b3f7eaa&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.406090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac35e5-4211-4326-ad2c-69fec01909e0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.406583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac35e5-4211-4326-ad2c-69fec01909e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.407077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac35e5-4211-4326-ad2c-69fec01909e0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.409364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 340/684] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:41.419058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.419726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.420203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.441181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:41.443039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:41.523913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 341/685] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:41.598742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8c78c-51e6-4f15-afea-225ed2cb38e8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.643804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.644410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.644744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.644961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.645744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:41.646047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.647563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:41.667125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 345/686] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:41.672787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.789696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f36ab-e19e-4644-aab7-2a84756e943d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:41.825919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.834821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:41.835730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:41.836344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:41.836620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:41.874686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:41.913497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:41.939351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:41.940827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.044639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0185d8-21bf-4c60-bebc-ad048d647df6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.118473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.119424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.119708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.119981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.120468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 342/687] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:42.123950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.140291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:42.146521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:42.147989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:42.148837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:42.150030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 346/688] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:42.155306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.181479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.264918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.302752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216e9ac-50db-4232-83fe-652e0e7b7729 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.318303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337a0db-ee77-4f4f-ac14-194df4f92240 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.359503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.360111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.360444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.360719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.361489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:42.361792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.363172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.378650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 347/689] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:42.385492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.396933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.405137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.405137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.405137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.419699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.421180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:42.451868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:42.453161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:42.454115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:42.454115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:42.454115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:42.454115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 343/690] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:42.458354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.515746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af172e1-cea6-49c1-85b4-dbc3d4e946ac None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.560218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.560860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.561165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.561448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.586783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.588241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.594940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.599029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:42.605407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:42.606925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:42.607884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:42.607884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:42.607884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:42.613534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 348/691] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:42.620848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.646574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f872c-deaa-4ae7-9b4c-f10ef60df34d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.708081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.708711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.708915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.709217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.709902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:42.710206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.711572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a8249-3a05-4e31-b311-33c31b3658e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.725514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 344/692] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:42.727050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006eaa6-9fd6-4459-9a74-3eda25c81ccf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.727655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006eaa6-9fd6-4459-9a74-3eda25c81ccf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.729087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006eaa6-9fd6-4459-9a74-3eda25c81ccf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.731600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 345/693] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:42.735541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006eaa6-9fd6-4459-9a74-3eda25c81ccf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.789166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174fba6-09ae-4d6d-b92f-bdcedab7c66e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.828415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.829440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.829988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.830469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.831345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:42.831903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.833529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704c3e2-2f6b-4457-85d0-98bd8ac7752b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.846592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 349/694] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:42.848237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.848732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.849261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.864321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006eaa6-9fd6-4459-9a74-3eda25c81ccf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:42.870879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:42.872089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:42.903144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:42.904139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:42.904628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:42.905110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:42.914284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:42.916239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:42.918649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:42.932669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:42.933864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:42.934782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:42.941721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 346/695] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:42.942738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:42.944770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:42.960737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 350/696] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:42.960737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.140108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db5ce3-97b7-43fb-bc8a-76b8f25178f8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.157427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21485bd7-fff1-4dc9-b44f-3910253abb0e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.188362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.189349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.189881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.190416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.191321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:43.191843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.194173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.207463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 351/697] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:43.214717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.243220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.244288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.244572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.244847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.257481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.258950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.269849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:43.283408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:43.284449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:43.285476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 347/698] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:43.291084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.333018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6fc46-2aa3-43fe-ab4b-5417e5765c4f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.355401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.373860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.374874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.375442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.375926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.376971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=6834988c33e94b9cb41df95e4bb1bd88) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:43.377559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.379340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.393147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 352/699] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/role_assignments?user.id=6834988c33e94b9cb41df95e4bb1bd88&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:43.394674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.400016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77348b-00e5-4d9f-922f-c6c2710fb018 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.441829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.442457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.442738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.443013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.443831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:43.444493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.446020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.484386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 348/700] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:43.492629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.563890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2fd12-34bc-416b-94cc-47d3550ef5c5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.578436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.600535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3b3d-9309-4366-b118-7937f6e8343c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.604077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.604955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.605477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.606446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.618575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.620656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.625783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.637692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:43.638234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:43.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:43.640043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:43.640043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:43.640043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:43.640212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.640212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.640212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.643425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 353/701] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:43.649851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.729495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.730966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259180f4-f3cf-49c4-bd97-11cdc290ac8e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.776437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6770f-8c69-4af6-b969-1ad87f644b79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.810907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.811599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.811794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.812069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.812721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:43.813026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.814633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820769-c4a6-4fbc-b7be-7ca7d15c6c9a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:43.827278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 354/702] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:43.828609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ec240-dada-4569-98b5-c5485592234c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.833573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ec240-dada-4569-98b5-c5485592234c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.834290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ec240-dada-4569-98b5-c5485592234c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.836036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 355/703] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:43.836782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ec240-dada-4569-98b5-c5485592234c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.862421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.868179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 349/704] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 07:01:43.869723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.870002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.871020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.896735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:43.898829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:43.913846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:43.953034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ec240-dada-4569-98b5-c5485592234c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:43.985167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 350/705] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:43.985167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:43.995271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:43.996107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:43.996553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:43.996982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:43.997941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:43.999623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe20cae-1de1-4f17-b44c-866bf9f1b9a3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.027667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 356/706] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:01:43 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:44.029335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0646-bd60-4f91-87e9-32e329379ce7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.029792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0646-bd60-4f91-87e9-32e329379ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.030214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0646-bd60-4f91-87e9-32e329379ce7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.032167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 357/707] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:44.038334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.039001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.039505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.069548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:44.071089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:44.213657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf082-f43e-40b0-9a9b-a9d59b434154 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.283873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.284442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.284724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.285000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.287535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 358/708] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:44.288679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.296897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.298691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.309674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.311813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:44.320247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:44.321303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:44.322414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:44.322414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:44.322414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:44.323947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 351/709] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:44.329273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.420962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e29a8-7654-46d7-9406-7a84befcd3ce None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.452125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22d7e7-9b27-40f4-acf1-65b3162524f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.469657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.470295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.470576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.470849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] PATH_INFO: `/v3/services/network` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.471447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.472809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.491589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.492238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.492519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.492794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.493521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:44.493776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.495410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:44.499836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:44.501154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.502021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 359/710] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:44.510096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.520775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 352/711] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:44.526434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.625359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.637268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.671729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d08d27-f2ad-476e-bc4b-23917304fc11 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.675495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8111bc-19ef-4892-975c-9e0c7093cf5a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.711125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.711745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.712028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.712302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.712940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:44.713275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.714708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.717375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.718031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.718310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.718587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.719896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 360/712] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:44.720750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.727678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.729039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:44.742124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:44.745330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 353/713] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:44.747579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.832912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297241b2-5393-4782-bd69-cd99f41bc70e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.873099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1c1d4-27f3-4d2d-b50d-e2a32773ac7b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:44.874219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.875159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.875496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.875708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.880673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.880673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:44.890580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:44.891627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:44.893925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 361/714] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:44.895522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:44.901704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.903140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:44.926898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:44.928488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:44.929273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:44.929728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:44.930814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:44.931719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:44.933833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:44.943158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 354/715] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:44.944252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:45.121056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcd67a-c3d1-4263-8013-0b41687bc2dc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:45.123344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c702fd0-21eb-4ca9-aa8f-3baa879d902e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:45.167931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:45.168554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.168841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.169254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.170440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:45.170855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:45.172486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:45.179359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 355/716] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:45.189343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:45.189931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:45.190548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.190823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.191091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.191788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:45.192089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:45.193684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:45.214503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 362/717] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:45.220256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:45.274869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:45.309669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:45.339040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316452ba-9076-4bcc-b03d-bdaf98eb23c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:45.380069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:45.381630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.382191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.383685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.385024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:45.385748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:45.387700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49c8a2-7031-43cc-b9ef-50a8215ede51 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:45.394002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 356/718] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:45.414467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c4da6-6ac9-4f51-b2c3-33443ce7eacc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:45.461400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:45.461943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.462224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.462498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.463240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:45.463547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:45.465009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7691-b213-4991-8f6d-33a4901d6aea None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:45.479006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 363/719] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:45 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:45.848452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6054cb-77f9-467a-b6fd-a0309c5be2bf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.849554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6054cb-77f9-467a-b6fd-a0309c5be2bf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.850092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6054cb-77f9-467a-b6fd-a0309c5be2bf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.851943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 357/720] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:45.857698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:45.858144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:45.858321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:45.886229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:45.887443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:45.968379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 364/721] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:46.308086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6054cb-77f9-467a-b6fd-a0309c5be2bf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:46.438748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6054cb-77f9-467a-b6fd-a0309c5be2bf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:46.450815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:46.488437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:46.489463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:46.489965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:46.490498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:46.504663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8ee58314-7f22-43cb-89c1-fe34790a5249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:46.513982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:46.514932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:46.515767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 358/722] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:46.523227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:46.596580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b587-a69f-4123-b001-922b8764e2e5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:46.596873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b587-a69f-4123-b001-922b8764e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:46.597036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b587-a69f-4123-b001-922b8764e2e5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:46.598403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 359/723] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:46.607159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:46.607439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:46.607714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:46.617491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:46.645687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:46.647413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:46.684291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8fce3-da67-4e93-b3aa-296738881980 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:46.719838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:46.720443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:46.720721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:46.720991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:46.721699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:46.722003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:46.723525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:46.733300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 365/724] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:46.738804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:46.770701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 360/725] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 07:01:46.842214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac3cb9-c748-452c-b87d-b08941ee9762 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:46.860590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:46.878131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:46.878757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:46.879033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:46.879302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:46.890200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:46.891640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:46.906876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:46.907885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:46.908806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:46.908806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:46.908806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:46.908806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 366/726] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:46.913991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.036817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:47.040330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de7001-de45-4dc6-9b47-33294d3aff86 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.070674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.104174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.104839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.105140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.105408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.106110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:47.106415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.109042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.137178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 361/727] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:47.158383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.265540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d58c08-4326-4a76-850e-bb750baf42c6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.285635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb484840-7677-4d4c-9357-1205158b23bb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.325351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.325982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.326264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.326533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.341446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.342895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.345237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.345871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.346146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.346423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:47.363190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:47.365981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:47.368203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 362/728] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:47.376334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:47.379122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:47.391353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:47.393553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:47.395588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 367/729] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:47.400405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.421652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:47.462301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:47.522959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad39c94-3e46-4f7b-a68e-0fb4f6a94f5e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.558127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5bae4-251e-4d58-9d38-958399749a2f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.571124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.571912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.572325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.572606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.573385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:47.573947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.576157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bafa67-cf1f-4b23-8eb9-4b154fae1d34 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.599512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.600367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 363/730] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:47.601763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb5e2-a1db-4425-95ac-eb4bb19e8c01 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.602049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb5e2-a1db-4425-95ac-eb4bb19e8c01 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.602324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb5e2-a1db-4425-95ac-eb4bb19e8c01 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.605239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.605594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.605942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.606686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:47.607072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.607567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 364/731] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:47.608372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb5e2-a1db-4425-95ac-eb4bb19e8c01 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.608624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65733d0a-e6c9-4d24-bb91-f6bb675aea0e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.620102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 368/732] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:47.621517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.622236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.622591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.646043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:47.647306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:47.708127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:47.734072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 369/733] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:47.735174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.750297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb5e2-a1db-4425-95ac-eb4bb19e8c01 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.783167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:47.801403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.802014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.802293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.802565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.888634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.890362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869583b5-4eee-40fb-8125-a8da59388f66 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.900598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256eb1-d63f-4523-8e2b-c531d1f0a5b8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:47.937441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:47.938075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.938358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.938635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.948719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:47.950191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d0da02a1-a0ad-49dd-9111-dcfafee01430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:47.965702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:47.966650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:47.967555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:47.967555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:47.967555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:47.967555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 370/734] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:47.968547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96cb0-30cf-4460-9770-72721d08e2e0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.968547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96cb0-30cf-4460-9770-72721d08e2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.968547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96cb0-30cf-4460-9770-72721d08e2e0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.970132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 371/735] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:47.971507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029cc8-771f-477b-a600-2f06c9895cf7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:47.971507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029cc8-771f-477b-a600-2f06c9895cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:47.971818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25029cc8-771f-477b-a600-2f06c9895cf7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:47.973044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 372/736] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:47.978156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25029cc8-771f-477b-a600-2f06c9895cf7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:47.978244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.005797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 365/737] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 07:01:48.005797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b64e-90c8-472e-b48e-68ccf1cde52f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.005797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b64e-90c8-472e-b48e-68ccf1cde52f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.005797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2b64e-90c8-472e-b48e-68ccf1cde52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.023279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-77a2b64e-90c8-472e-b48e-68ccf1cde52f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:48.024553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-77a2b64e-90c8-472e-b48e-68ccf1cde52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:48.060952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.089552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25029cc8-771f-477b-a600-2f06c9895cf7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.094196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 366/738] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:48.095497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.095785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.096057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.120590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:48.121884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:48.123700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.124294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.124567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.124836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.125498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:48.125798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.127149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.143187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 373/739] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:48.143187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.198697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 367/740] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 07:01:48.199508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.251262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.260363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367b2e7-f16d-4f99-8de0-afd032a1816a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.294994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c50be5-252b-4777-9589-124990d277eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.295619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c50be5-252b-4777-9589-124990d277eb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.295891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c50be5-252b-4777-9589-124990d277eb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.296157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c50be5-252b-4777-9589-124990d277eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.298162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.308703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c50be5-252b-4777-9589-124990d277eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.310180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c50be5-252b-4777-9589-124990d277eb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-52c50be5-252b-4777-9589-124990d277eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:48.325156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:48.326106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:48.327024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:48.327024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:48.327024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:48.327024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 374/741] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:48.327024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c50be5-252b-4777-9589-124990d277eb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.337149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c03b-fe85-468f-92d9-40f0fae1bcf4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.379567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.380172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.380443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.380707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.381902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.383259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.405430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 368/742] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:01:48 2022] POST /identity/v3/services => generated 240 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:48.405430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.443385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c50be5-252b-4777-9589-124990d277eb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.498377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.499005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.499284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.499557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.508074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.509596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:48.523454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:48.524690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:01:48.525775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:48.528392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 375/743] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:48.529107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.535148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.540611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4607301-bb5d-41fc-ad46-e54bf13236ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.580415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.583287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.583912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.584263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.585054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.585818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:48.586057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.587820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.600187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 369/744] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:48.601009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.651868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f189d-a4a6-4823-b2ea-69ad8c9ad5ee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.688008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.688633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.688918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.689224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.700647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:48.704919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e65ca-37de-4997-938c-deebf8158857 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:48.706072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:48.707031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 376/745] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:48.707984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.747859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.748900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.749433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.749930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.750776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.752364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.770830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 370/746] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:01:48 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:48.782303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.812250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.824390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611e139-e59d-433a-b728-74acdb658f91 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.826437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:48.871706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.872324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.872606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.872878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.873659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:48.873963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.875420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.896105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 377/747] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:48.901598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:48.906757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5ae88-05d8-42a4-ab6a-9254bc79d6fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:48.950179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:48.950179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:48.950179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:48.950179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:48.958124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:48.959275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:48.972093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:48.973000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:48.973939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 371/748] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:48.974448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.030120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9cb9ab-4a01-427b-ba72-62b868725b0c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.073425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.081241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.081870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.082130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.082402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.083087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:49.083371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.084771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.093843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 378/749] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:49.094641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.105455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9c368-afbd-452c-a9ff-c52c37da9375 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.114762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.150196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.150811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.151140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.151359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.211073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa23223-85eb-4657-9006-78ede5019535 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.223190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.224718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.251429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.252393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.252876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.253432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.254480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:49.254977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.256928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.270235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 379/750] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:49.271505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.305728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.333439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 372/751] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:01:49.334368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.363467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.389136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1217c-10f3-4724-9db4-48d9025ef145 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.426466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.427548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.428059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.428535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.440065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.440205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc7ce6-60da-4e84-be72-b1b70c412cca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.442038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:49.457490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:49.458469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:49.459918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:49.459918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:49.459918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:49.462139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 380/752] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:49.463179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.477013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.477667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.478269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.478402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.479028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=6834988c33e94b9cb41df95e4bb1bd88) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:49.479416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.480976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.486961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 373/753] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/role_assignments?user.id=6834988c33e94b9cb41df95e4bb1bd88&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:49.487668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.536774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.582434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a012d-14a4-4717-be5b-d3590a8cf7c6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.602385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0442594-9ba8-4557-a58e-681ed08dd606 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.616029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.622887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.623900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.624376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.624869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.637222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.639448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.641222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.641851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.642130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.642646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.643172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:49.643488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.645000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:49.658139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:49.659135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:49.660030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:49.660030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:49.660030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:49.660302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 374/754] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:49.662417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 381/755] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:49.664637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.674268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.823997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca9331-885a-4664-802d-0303751f4130 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.855193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09649fc8-0612-4997-8cc2-7e68d9ed1cda None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:49.856706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:49.861545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.862833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.863112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.863386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.864033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:49.864344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.865894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.890473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 382/756] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:49.895652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:49.896290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:49.896568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:49.896731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:49.897459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:49.905095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:49.906589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:49.918748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:49.919716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:49.920651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 375/757] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:49.926032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.034629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.045339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a4388-bc52-4753-8821-05ec63b92225 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.117785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.118513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.118685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.120005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.120752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:50.122205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.127271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.148497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9d8e9-6f72-4b47-83c2-3d05afc8129f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.166224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 376/758] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:50.168943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.213688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.219083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.219728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.220006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.220276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.232647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.235348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:50.263041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:50.265093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:50.267200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 383/759] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:50.270597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.380327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963b61f-4206-4c0d-b8d6-2fff61e92008 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.454123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78189eb6-8e1a-4102-92d2-aea78e296d8d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.455629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.458373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.459471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.460096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.465614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.490527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.492878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.500318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.500923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.501244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.501518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.502210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:50.503016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.504898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:50.511030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:50.511962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:50.512841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:50.512841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:50.512841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:50.514922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 377/760] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:50.520677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.535398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 384/761] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:50.543076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.611081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.651619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4b46-d448-4655-b3b7-3b19fdc87342 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.694937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dafca-7015-4575-92b7-452787839c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.695552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dafca-7015-4575-92b7-452787839c6e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.695828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dafca-7015-4575-92b7-452787839c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.696097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dafca-7015-4575-92b7-452787839c6e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.696989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dafca-7015-4575-92b7-452787839c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:50.697318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dafca-7015-4575-92b7-452787839c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.699568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dafca-7015-4575-92b7-452787839c6e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.701556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232de65-964e-433d-b00a-8edfce50cd92 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.716154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 378/762] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:50.717639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd72a3-c179-43fa-8b12-f8f1049b2a0b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.717925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd72a3-c179-43fa-8b12-f8f1049b2a0b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.718577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd72a3-c179-43fa-8b12-f8f1049b2a0b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.724320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 379/763] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:50.725642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd72a3-c179-43fa-8b12-f8f1049b2a0b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:50.752313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.753066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.753383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.753714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.754761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=d253c4509e954c11927f3f1f4563a5ab) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:50.755072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:50.756575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224158d-053d-49a1-80bb-9452ea73c442 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:50.765211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 385/764] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:50 2022] GET /identity/v3/role_assignments?user.id=d253c4509e954c11927f3f1f4563a5ab&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:50.766815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5133-12b3-478f-a25c-36d1680d0c58 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.767302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5133-12b3-478f-a25c-36d1680d0c58 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.767785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5133-12b3-478f-a25c-36d1680d0c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.806189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d25f5133-12b3-478f-a25c-36d1680d0c58 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:50.807410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d25f5133-12b3-478f-a25c-36d1680d0c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:50.848514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.911741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd72a3-c179-43fa-8b12-f8f1049b2a0b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:50.921469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 386/765] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:50.923155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660ab0-e619-4153-8f1e-fd08e26b09b3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.923672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660ab0-e619-4153-8f1e-fd08e26b09b3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.924197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8660ab0-e619-4153-8f1e-fd08e26b09b3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.929988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 387/766] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:50.932380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.933310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.933589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:50.953539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:50.954744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:50.971705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:50.984438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:50.985396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:50.985873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:50.986477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.001947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.004338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:51.029691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:51.030641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:51.031567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:51.036977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 380/767] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:51.041710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.077061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 388/768] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:51.179991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d41f5-a681-4316-bff5-5d2a4e07349f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.213065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.229392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.230617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.230893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.231160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.231858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:51.232159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.233629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.261377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 381/769] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:51.267963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.352337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:51.364523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:51.373361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0fee9-2b84-489e-b4dc-8590ce99ce1e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.411942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.414032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.414310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.414582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.415167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.416537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:51.426766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:51.427760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:51.429193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 389/770] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:51.434976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.476490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f017-4e27-4c04-b364-6b0d13e586cd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.524617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.526222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.526703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.527171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.530141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=3d44b8c931e047408a2d9d4baa490806, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.532112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.550751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 382/771] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:51 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:51.551928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.593049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2a72a-4ecb-40b7-b97f-c3e5658a24dc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.680012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.680649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.680929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.681224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.681876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:51.682263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.683561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.697031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4df8f-1cbe-4178-97f5-8bcd58c3923a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.698153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 390/772] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:51.704125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.719919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:51.728416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:51.751997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.752702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.752982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.753298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.753873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.755252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c1d443c6-0d9a-4575-ad8e-dad035cf38aa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:51.768913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:51.770080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:51.771465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 383/773] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:51.772831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754e788-030f-485f-b9ce-15e9b33178ff None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.772831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754e788-030f-485f-b9ce-15e9b33178ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.772831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754e788-030f-485f-b9ce-15e9b33178ff None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.774043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 384/774] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:51.777987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754e788-030f-485f-b9ce-15e9b33178ff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.833750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d29936-81fc-49c1-8fd2-ed68b874680d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.874223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.875255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.875773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.876280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.877173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] RBAC: Adding query filter params (type=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:51.877792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.879419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4dd7f-8acf-44a1-9de0-d7e05acc53ee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.886760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 391/775] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:51.888269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.888759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.889294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.909961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:51.911524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:51.913050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754e788-030f-485f-b9ce-15e9b33178ff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:51.957195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:51.957719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:51.957993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:51.958261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:51.958890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:51.959734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:51.961799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:51.973067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 385/776] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:51.974104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:51.995707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 392/777] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:51.997018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.011621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.030185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.116947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bc739-6c5c-43ac-aae1-d74f57ef002e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.164844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0c8f8-bc50-41d5-adbc-0c4c7b8ca18d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.182949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.183549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.183826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.184092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.186142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] RBAC: Adding query filter params (interface=public, service_id=df6071dcc986487da646013ae3eac5ec, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:52.186480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.187871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.201457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 386/778] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/endpoints?service_id=df6071dcc986487da646013ae3eac5ec&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:52.202738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.243537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.244373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.244654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.244919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.245569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:52.245870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.247226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.260620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 393/779] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:52.265929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.362330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ac14-9d45-488a-87b7-65189e832776 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.365905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.367302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.404199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.405195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.405670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.406139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.406961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.408028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc088c-05ca-4fb3-b48f-c2ef742f96cd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.408821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.415682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 387/780] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:52.416683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.470774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.471767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.472245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.472714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.473882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] RBAC: Adding query filter params (interface=public, service_id=c66b2db8ec22426fa0fdfeb994bf6d0c, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:52.474444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.476770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.486597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 394/781] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/endpoints?service_id=c66b2db8ec22426fa0fdfeb994bf6d0c&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:52.492944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.624538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd9cce-dbf5-4226-a0e9-fa245cb99d8e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.651575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e94a2b-8dc9-4633-b61e-ff7035753b16 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.663439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.664076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.664348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.664674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.676463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:52.681931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:52.682873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:52.683724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:52.686089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 388/782] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:52.689883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.692466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.693091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.693392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.693713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.694384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.696273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.698500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.702404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 395/783] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:52.729407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:52.828755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd4d7a-89e4-4062-b232-6f154d5569c7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:52.867046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:52.868175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:52.868684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:52.869432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:52.870303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:52.870827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:52.872429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37620b-b683-4797-8371-6b25c2ba8e3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:52.883023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 389/784] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:52.894240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:52.997852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f125-cdfb-4c36-927e-37f4cb39d783 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.027932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a9159-a7dd-49f7-8585-e8d022ee8373 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.028248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a9159-a7dd-49f7-8585-e8d022ee8373 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.028497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a9159-a7dd-49f7-8585-e8d022ee8373 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.032836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 390/785] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:53.035327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.035961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.036236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.036507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.037066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.037388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.037663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.052038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:53.053564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:53.073039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:53.078731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:53.080296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:53.081233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:53.082135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:53.082135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:53.082135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:53.082135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 396/786] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:53.091888 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.177601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:53.212177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 391/787] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:01:53.277612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948bec3-5e6a-45be-87da-d798dcbee510 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.335247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.335856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.336135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.336408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.337054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:53.337380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:53.338748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:53.352026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 397/788] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:53.357440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.397595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:53.501806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.501980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8118c-13b2-4a7c-8924-093ad65e2d26 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.536415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.537047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.537888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.538158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.545386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:53.546843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:53.559332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:53.560321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:53.562046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 392/789] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:53.566834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.651522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:53.670672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a6edc-59e7-406d-abfa-3fc7ee4fcf59 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.675010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e083d-ae18-4201-8c1f-df30dab801f7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.691049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:53.712453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.713249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.713536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.713814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.726005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:53.731089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:53.732052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:53.733370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 398/790] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:53.741308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.759889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.760426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.760701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.760967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.761721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:53.762020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:53.764991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2478-d80b-4f87-8be7-cb66627ab9ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:53.789582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 393/791] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:53.791405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f760a-6f9a-48f9-9569-92577f8969cc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.791904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f760a-6f9a-48f9-9569-92577f8969cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.792634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f760a-6f9a-48f9-9569-92577f8969cc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.795239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 394/792] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:53.796149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f760a-6f9a-48f9-9569-92577f8969cc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:53.878495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a591-e8d3-4eea-b8f4-36142e686c46 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.925573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:53.926561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f760a-6f9a-48f9-9569-92577f8969cc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:53.926774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.927225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.927697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.928874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:53.930425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:53.932075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0097be3-b61b-4c27-9209-f7665a69f6a0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:53.942836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 399/793] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:53.944596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:53.945098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:53.945578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:53.966104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:53.967679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:53.999812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.048246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 400/794] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:54.049420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.135164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.136142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.136284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.136557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.150077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.159653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.161298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.167511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cd629-83c9-4019-893c-0df9d4178634 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:54.195668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:54.196673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:54.197628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:54.197628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:54.197628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:54.197628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 395/795] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:54.202736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.213394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.216266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.217009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.217495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.230189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.232293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:54.267868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:54.268871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:54.269792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:54.269792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:54.269792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:54.272124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 401/796] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:54.277836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.304733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.394993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eb307-4d06-47ca-a02f-0eeb11d2fda8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.431971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.432891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.433532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.434121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.434968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:54.435476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.437290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.450634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 402/797] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:54.465579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.506475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bdfec-8e93-4125-9672-aed0ce99ac3b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.542300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbd240-337a-480d-b870-336325cce007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.542909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbd240-337a-480d-b870-336325cce007 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.543183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbd240-337a-480d-b870-336325cce007 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.543453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbd240-337a-480d-b870-336325cce007 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.544087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbd240-337a-480d-b870-336325cce007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:54.544381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbd240-337a-480d-b870-336325cce007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.545853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbd240-337a-480d-b870-336325cce007 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.563051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 396/798] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:54.563874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbd240-337a-480d-b870-336325cce007 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.575146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.616258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.624572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0707d-51e1-4e4f-a8ad-f8b9310e0312 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.670239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbd240-337a-480d-b870-336325cce007 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.681393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.683162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.685061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.686142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.705139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.705791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.706123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.706403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.714823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.716371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.719314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:54.734462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:54.735458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:01:54.736389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:54.740757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 397/799] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:54.741639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:54.745732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:54.746626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:54.747488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:54.754117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 403/800] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:54.754117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.895393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ca844-9008-4744-90d9-c8b1120749e6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.901422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52a200-f10f-47b1-b9f4-3ee6acc07363 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:54.911928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.934672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.935275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.935567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.935837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.936563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:54.936864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.938337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.940809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:54.941776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:54.942815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.943031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.943367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.950077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:54.951511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:54.959711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 404/801] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-68d0fabc-a982-43f0-9b26-6cdf2820de2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:54.963272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:54.964173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:54.965066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 398/802] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:54.965982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:54.966053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cb4f-fe74-47d6-9a64-2f08722ad8f5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:54.966053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cb4f-fe74-47d6-9a64-2f08722ad8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:54.966362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88cb4f-fe74-47d6-9a64-2f08722ad8f5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:54.969296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 399/803] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:54.969296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88cb4f-fe74-47d6-9a64-2f08722ad8f5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.077039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88cb4f-fe74-47d6-9a64-2f08722ad8f5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.093707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f2f5-a15f-4d02-bdc6-76a6d37675df None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.111882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.112503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.112804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.113038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.124346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.125828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.133327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.133943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.134223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.134497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.135172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:55.135477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.136832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de453169-594a-4289-a17e-7bb15d9c3ec7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:55.140701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:55.141841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:55.142727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:55.142727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:55.142727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:55.142727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 400/804] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:55.142952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.148866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 405/805] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:55.150029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728b20-09af-43df-a41e-a9e88fa097f9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.150311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728b20-09af-43df-a41e-a9e88fa097f9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.150582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84728b20-09af-43df-a41e-a9e88fa097f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.167499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.168000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-84728b20-09af-43df-a41e-a9e88fa097f9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:55.169353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84728b20-09af-43df-a41e-a9e88fa097f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:55.219777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.236774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 406/806] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:55.238039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753b0e-a252-4890-b6e8-889eb2a6d99a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.238371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753b0e-a252-4890-b6e8-889eb2a6d99a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.238639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753b0e-a252-4890-b6e8-889eb2a6d99a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.239872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 407/807] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:55.240593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06753b0e-a252-4890-b6e8-889eb2a6d99a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.244336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b519cb-8508-4021-94d5-634db8d09eb1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.281860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.282745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.283030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.283300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.283991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:55.284295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.285761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.303447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 401/808] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:55.304312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.360707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06753b0e-a252-4890-b6e8-889eb2a6d99a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.388332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.397841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.398636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.399228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.399644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.400446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:55.400755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.402163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9ee2-ee1b-4184-85cf-139e35f7b30e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.407537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f576fa8-5eb7-4648-b408-41f290ce201d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.416599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 408/809] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:55.417988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.418339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.418621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.436325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:55.437517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:55.448011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.448879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.449395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.449766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.461285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.463058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:55.479580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:55.480527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:55.486561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:55.486561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:55.486561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:55.486811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.486901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 402/810] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:55.486901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.520602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 409/811] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:55.520602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.592947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2520dd-198e-43ec-9461-3c007361d050 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.620003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb70ea-fa7e-4561-b7d4-86dfaf947c0d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.628214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.628860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.629154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.629434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/6834988c33e94b9cb41df95e4bb1bd88/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.631318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=6834988c33e94b9cb41df95e4bb1bd88, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.632748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.635025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.657233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 403/812] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:55 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/6834988c33e94b9cb41df95e4bb1bd88/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:55.657233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.668246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.669127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.669482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.669822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.678057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.679809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:55.693078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:55.694290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:55.695204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 410/813] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:55.695705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.738326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.802009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa3d4b-c8c0-4b60-9ee0-c7c33c6d88b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.828387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb782e-e930-49b5-872a-c8907f5eda55 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.842926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.843543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.843825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.844106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.844743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:55.845044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.846452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.860138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 404/814] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:55.860956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.870213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:55.870924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:55.871270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:55.871611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] PATH_INFO: `/v3/services/network` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:55.872265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:55.873719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:55.883720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:55.884934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:55.885850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 411/815] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:55.886664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:55.927257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.976163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a585de0-ed11-4814-a54e-32acc884fe93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:55.991318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:55.995790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b60db-ae4e-4c26-a79c-217a6956c612 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.015027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.015626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.015906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.016180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.016863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.017190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.018615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.030948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 405/816] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.031570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.043952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.045086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.045558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.046375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.047764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.049853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:56.062276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:56.063390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 07:01:56.064424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:56.066757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 412/817] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Sep 1 07:01:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:56.067683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.140308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6d0a0-2762-4a4f-a943-7661c7bc1d11 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.154526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.175256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4835f3-d4c5-4dc4-8484-fe6e94cad6bb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.179532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.180101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.180380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.180654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.189542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.190984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:56.202617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:56.203623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 406/818] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:56.205462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.217061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.218758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.219031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.219306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.219932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.220461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.221648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.226637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 413/819] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.227409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.262188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.317772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53602-6549-4fa0-9809-aa48ce09cc47 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.337036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9d1c-cd3c-4703-9cce-5c43b39bac2c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.352483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.353179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.353468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.353745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/d253c4509e954c11927f3f1f4563a5ab/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.370428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=d253c4509e954c11927f3f1f4563a5ab, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.371884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.372120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.372721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.373000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.373305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.373930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.374245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.375586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.385821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 414/820] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.393764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 407/821] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:01:56 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/d253c4509e954c11927f3f1f4563a5ab/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:01:56.393955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.394065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.426802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.510226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.521565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d0bae-9395-4d76-9326-62d6035b551c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.546519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de2ab0-b86d-4595-b65a-6670a90cb6f3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.560792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.562372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.562801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.563220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.564914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.565390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.566977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.587075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 415/822] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.587926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.592120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.592997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.593582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.593753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.594484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.594797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.596175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.602969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 408/823] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.603773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.710202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559524b1-3660-4f08-9a21-0ec61870989d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.727280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.745616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157fe2c-fa5d-4e8a-b450-be4f83e84688 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.756465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.757286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.757568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.757836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.758483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:56.758783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.760191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.769523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 416/824] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:56.770453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.788083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:56.793489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.794109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.795592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.795887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.885200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587261b-525e-473d-b167-bdb1c4ff2fee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:56.890728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.892681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.920943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:56.921628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:56.921901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:56.922169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:56.922871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:56.924272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:56.951327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 417/825] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:01:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:56.952539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:56.996631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:57.011299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 409/826] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:01:57.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:57.036103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:57.080223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106dfd0-086d-46b4-889a-f065ca4c736d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.119520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.120357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.120690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.121051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.121831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.123313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.133990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d898d06-03e8-442a-926d-699d2014f9df None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.146429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 418/827] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:01:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:01:57.193132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.193839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.194116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.194387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.206339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.209843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:57.228185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:57.229262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:57.230168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:57.230168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:57.230168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:57.230168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 410/828] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:57.236428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:57.388817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b79dac-5a7e-445b-a734-e7559a9f659c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.414690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:57.427407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.428037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.428314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.428580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.429332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:57.429808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.431444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.444512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 419/829] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:57.449695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:57.485302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:57.555957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842ca4-43a4-4112-813f-961caf1c61c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.590970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.591606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.591884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.592160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.599186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.600678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-717fa63d-5c3c-4d5f-84e3-1e83aee196fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:01:57.612215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:57.613360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:57.614361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 411/830] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:57.621678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:57.669291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f498c7f-3208-44d8-b77e-25d030c152ac None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.669475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f498c7f-3208-44d8-b77e-25d030c152ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.669754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f498c7f-3208-44d8-b77e-25d030c152ac None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.671925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 412/831] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:57.676957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.677277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.677552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.696812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:57.698034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:57.728807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6f40-dc42-4eca-b89d-bd590e2e587e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.778189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.779134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.779709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.780179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.785151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:57.785270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 413/832] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:01:57.788486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.788486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.799826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 420/833] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:57.805960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:57.833402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:57.907666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99b0f6-c2b0-40d8-ab0a-1acb68d5e8fa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:57.943152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:57.944228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:57.944504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:57.944768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:57.945514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:57.945815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:57.947239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:57.953415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 414/834] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:01:57 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:58.060398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:58.107415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:58.167704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767b83-739e-449c-b0f3-f3f9c793e1a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:58.206655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:58.207512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.207797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.208204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.220012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:58.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:58.226262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:58.227117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 421/835] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:58.234251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:58.334155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:58.345404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab672d3-17a5-4fc6-a748-d049eaefadba None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:58.381915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:58.382829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.383282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.383758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.384660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:58.385171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:58.386758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:58.400934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 415/836] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:58.403343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:58.510934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ac35-695f-46e3-bd46-864a1a105500 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:58.512698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:58.547933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:58.548872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.549424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.549875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.561595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:58.563039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8bc0bbc2-71ee-440d-a5fc-51f2b3ff82c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:58.578573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:58.579564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:58.580486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:58.580486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:58.580486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:58.580486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 422/837] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:58.587030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:58.652733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb76a-65d0-4ea5-8666-330c5ade2aa5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.653084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb76a-65d0-4ea5-8666-330c5ade2aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.653305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56eb76a-65d0-4ea5-8666-330c5ade2aa5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.654905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 423/838] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:01:58.659678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.659958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.660227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.679376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:58.680562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:58.695286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7e31d-be0e-4c7b-97e6-f7892013a4eb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:58.731252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:58.732236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.732731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.733241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.734143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:58.734641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:58.736271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:58.749929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 416/839] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:58.755514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:58.767779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 424/840] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:01:58.774166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:58.861400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43559feb-17b9-4db3-981d-2069e671e618 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:58.896641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:58.897625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:58.898108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:58.898578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:58.906117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:58.907817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:01:58.920274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:58.921275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 07:01:58.922237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:58.924674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 417/841] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:58.931045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.019908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.036466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93490e-17cf-4a41-a34c-ba25b3e82bb0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.051146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.076816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.077459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.077742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.078021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.078862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:59.079169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.080719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.100476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 425/842] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Sep 1 07:01:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:59.106400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.134926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.175767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb77f1-f09b-476b-ab87-9333cf3b9b7d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.222386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.223030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.223310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.223590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.234481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:01:59.237335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc57397-5385-40ff-a533-1a377cf5b2f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e2cdbe7c-8a6e-4ef2-aed5-f88a4390b04a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:01:59.241054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:59.242089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:01:59.242945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:59.247293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 418/843] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:01:59.248427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54e84-b327-4206-b933-010ec6ddcadd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.248720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54e84-b327-4206-b933-010ec6ddcadd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.248997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b54e84-b327-4206-b933-010ec6ddcadd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.253178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 419/844] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:01:59.253178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b54e84-b327-4206-b933-010ec6ddcadd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.279979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.280589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.280864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.281253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.294111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.357802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.359253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.361019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b54e84-b327-4206-b933-010ec6ddcadd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.395527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.401687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.402808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.403180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.403547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.404649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:59.405047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.406831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e6596-ea36-41f1-bd46-e3b84d3212a2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.419635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 420/845] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:59.419635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.419635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.419635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.434716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:01:59.435943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:01:59.478311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 426/846] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 07:01:59.479258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.513878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 421/847] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:01:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:01:59.514937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.534916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.597396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755cd53-3fc4-409a-ac0f-08da1fe93b53 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.634533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.635158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.635432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.635699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.640512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa427dd-4742-4d1e-966e-069842479eb6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.646477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.648641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:59.664362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:59.665387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:59.666289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:59.666289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:59.666289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:01:59.670362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 427/848] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:59.671154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.674492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.689666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.690794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.691246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.691907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.706253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.707789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:01:59.723163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:01:59.724146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:01:59.725453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:01:59.725453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:01:59.725453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:01:59.725453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 422/849] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:01:59.729799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.794463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.803218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9ecb9-9823-46b8-9dd2-e24f3ae51c2d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.840709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.841357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.841630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.841906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.842537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:59.842831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.844181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.856651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 428/850] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:01:59.857512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.871893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964b5f-cb95-4942-aa25-959737cc27f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:01:59.907474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:01:59.908091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:01:59.908369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:01:59.908648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:01:59.909322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:01:59.909632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:01:59.911009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:01:59.923219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 423/851] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:01:59.924342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:01:59.964635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:01:59.965636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b2443-bdf0-454d-a341-4f0186445529 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.004027 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.004652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.004928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.005233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.005816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.007741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:00.017233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:00.018223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:00.019057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 429/852] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:00.019861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.041217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.043751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3a05a-bb2d-45e7-94ed-8e1e50c51a56 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.082782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.083321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.083596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.083878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.091008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.092443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:00.103836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:00.104932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 424/853] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:01:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:00.106296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.126907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e963e2-8cdb-4367-b55f-24467b627f1f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.160038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.160653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.160930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.161246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.167664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.169083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-31cd5a54-6520-4aeb-acb8-37b11b28e871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:00.181406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:00.182456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:00.183408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 430/854] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:00.183995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92bc70-d71f-4dad-ada5-cbec6e660e8f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.184273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92bc70-d71f-4dad-ada5-cbec6e660e8f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.184542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92bc70-d71f-4dad-ada5-cbec6e660e8f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.185846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 431/855] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:00.186522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92bc70-d71f-4dad-ada5-cbec6e660e8f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.204310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.211633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dfb67-81b8-419b-8e5e-74e505f7a5b3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.246135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.246740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.247019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.247291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.247910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:00.248210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.249637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.254966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 425/856] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:00.255733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.272546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.296468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92bc70-d71f-4dad-ada5-cbec6e660e8f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.329709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.330307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.330582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.330852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.331572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:00.331872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.333355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334be7bf-3320-4374-810d-5b7468798fc6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.350601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 432/857] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:00.351826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.352025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.352309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.360171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a7676-4fb6-408a-b721-cb9b8faf3415 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.369850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:00.371030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:00.394064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.394692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.394968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.395237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.395916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:00.396221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.397679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.409095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 426/858] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:00.409963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.435302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.439620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 433/859] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:00.440453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.506757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.519693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e0fc-0a01-4f3a-b6a4-2077aff2017b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.543231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628baed-bae6-4540-978c-8704274e8b7b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.556171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.556896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.557331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.557699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.558456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:00.558855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.560318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.577150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 427/860] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:00.577150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.593511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.594152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.594569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.595024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.675314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.676935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.699143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.719872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252609-adaa-4cec-808c-700cc2cca00d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.757332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:00.767016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.767753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.768134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.768506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.769464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=6834988c33e94b9cb41df95e4bb1bd88) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:00.769775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.771537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.777609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 428/861] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/role_assignments?user.id=6834988c33e94b9cb41df95e4bb1bd88&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:00.778275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.800411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 434/862] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:00.802483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.898610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a413-ffa6-4d7b-ac5c-ea05b5b62704 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.920371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcec8f-727b-47c8-ae6f-e361e6c0d73a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:00.939298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.939945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.940227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.940502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.941160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.943273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:00.954939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:00.955932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:00.956834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:00.959609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 429/863] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:00.963778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:00.964803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:00.965077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:00.965333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:00.965426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:00.976745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:00.978483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:00.985610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:01.000603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:01.001670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:01.006151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:01.006151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:01.006151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:01.006151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 435/864] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:01.007851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:01.039895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:01.089311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00dc8a-ab67-43da-9a33-5f6cac1c8cc8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.118312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730d72-cfd3-4ba3-879e-0e7c604ef4a6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.127322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.127823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.128206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.128579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.129345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:01.130104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.131658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.143128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 430/865] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:01.144224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:01.156967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.157838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.158410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.158858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.159674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:01.160155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.161849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.175106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 436/866] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:01.180879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:01.233397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:01.263954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79213e2-8aa8-4d84-9e7f-df63fe2a7599 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.298559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:01.308624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.309398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.309577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.309834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.310466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:01.310772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.312156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.317001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 431/867] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:01.326717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c382e-b4bd-4bd1-8248-a44eee1fe0ac None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.370247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.371945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.372536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.373250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.381622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.383439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:01.397243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:01.398201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:01.399090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:01.407108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 437/868] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:01.408823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:01.517698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07025-ee78-428c-9811-90b161c5387b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.553303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.553916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.554194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.554468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.555160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:01.555464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.557229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a204-a8d3-41d7-84fd-b491ad52242c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.566678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:01.570395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 432/869] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:01.576296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:01.679338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887a77e-6ebb-4ddb-9773-abac3f794e7c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:01.716388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:01.717520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:01.718011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:01.718475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:01.719393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=d253c4509e954c11927f3f1f4563a5ab) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:01.719891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:01.721532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55052116-e756-4d9a-a855-87313cc676b3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:01.733244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 438/870] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:01 2022] GET /identity/v3/role_assignments?user.id=d253c4509e954c11927f3f1f4563a5ab&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:02.539746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b2589-4287-4701-8f39-18a7e683fd53 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:02.540308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b2589-4287-4701-8f39-18a7e683fd53 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:02.540308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b2589-4287-4701-8f39-18a7e683fd53 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:02.541792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 433/871] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:02.547067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc8649-d8a2-4306-b361-124fb35bd5d2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:02.547578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc8649-d8a2-4306-b361-124fb35bd5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:02.548059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc8649-d8a2-4306-b361-124fb35bd5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:02.555171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:02.567654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d3cc8649-d8a2-4306-b361-124fb35bd5d2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:02.568861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3cc8649-d8a2-4306-b361-124fb35bd5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:02.637571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 439/872] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:02.840377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa9b5-ae8e-40b0-8615-04219e323fc7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:02.840677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa9b5-ae8e-40b0-8615-04219e323fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:02.840880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eaa9b5-ae8e-40b0-8615-04219e323fc7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:02.842492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 434/873] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:02.847632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:02.847927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:02.848206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:02.867600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:02.868797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:02.912740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaa9b5-ae8e-40b0-8615-04219e323fc7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:02.943386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 440/874] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:03.027387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eaa9b5-ae8e-40b0-8615-04219e323fc7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:03.063241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:03.063851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.064125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.064395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.065057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:03.066449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f98ce-4fb2-4463-a146-c4c25f6318a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:03.081976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 435/875] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:02 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:03.218064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:03.282218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:03.322324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab23b-d732-499f-b322-3bc2316cdae8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:03.324081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fa1b-7144-4976-9ade-4b2beca013b9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.324369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fa1b-7144-4976-9ade-4b2beca013b9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.324759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8fa1b-7144-4976-9ade-4b2beca013b9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.326390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 436/876] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:03.331416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.331704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.331983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.352827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:03.354123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:03.382641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:03.383295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.383588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.383850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.384441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:03.385895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:03.397146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:03.398153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:03.399007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 441/877] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:03.405589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:03.437349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:03.445282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 437/878] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:03.523116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff59ce-d3cf-44b0-95a6-c5c78c5a193b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:03.550104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:03.561803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:03.562849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.563434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.564098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.564733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:03.565039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:03.566445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348a102-5197-4965-ae77-d2e931fa24a9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:03.571990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 442/879] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:03.577590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:03.682504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448a1b-8660-4a10-b9e6-34361101fc62 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:03.717455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:03.718158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.718438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.718706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.719336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:03.719634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:03.721008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1beef3-16c2-4908-a07a-957f822e7992 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:03.725906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 438/880] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:03.736032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e87c4-95cc-4045-9051-01c30edf8adf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.736548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e87c4-95cc-4045-9051-01c30edf8adf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.736625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e87c4-95cc-4045-9051-01c30edf8adf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.738197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 443/881] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:03.744931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.745570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.746062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.769148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:03.769609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:03.779621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e87c4-95cc-4045-9051-01c30edf8adf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:03.835209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:03.861564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 439/882] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:03.898664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e87c4-95cc-4045-9051-01c30edf8adf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:03.934387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:03.934990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:03.935271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:03.935545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:03.936211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:03.937661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:03.957931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 444/883] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:03 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:02:04.144646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:04.256709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2d667-9cf3-4864-86e9-9292b3ff9e2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:04.298370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:04.299011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:04.299287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:04.299561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:04.310653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:04.315539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:04.316467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:04.317361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 440/884] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:04.322605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:04.373683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:04.434022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b504b49-ee55-44be-ac55-09716c91eea7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:04.473013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:04.473655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:04.473934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:04.474208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:04.475491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:04.475794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:04.478245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:04.487189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 445/885] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:04.492814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:04.536379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:04.598989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b2744-a880-45af-b722-445e5e404df9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:04.634672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:04.635310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:04.635590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:04.635865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:04.646366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:04.648473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:04.667952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:04.668921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:04.669870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:04.669870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:04.669870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:04.669870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 441/886] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:04.674862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:04.787128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cf8a3-1033-44fc-9652-2f074e801fa3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:04.796990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:04.825864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:04.826476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:04.826793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:04.827075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:04.827719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:04.828083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:04.829478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:04.842761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 446/887] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:04.848511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:04.947818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:04.958437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bcb7d-2a3c-4793-b43e-c515a3e91bbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:04.994100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:04.994997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:04.995273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:04.995540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.003410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.004849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:05.016728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:05.018598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:05.019949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 442/888] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:02:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:05.024238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.132501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04d514-0728-4d18-b8e4-2547f9d4d245 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.172425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.173028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.173336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.173613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.174343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:05.174649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.176089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119adca-0f8c-4bc3-a44f-89fb793a8354 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.194162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 447/889] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:05.199930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.229166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2e7f-c8b1-4e8f-8e87-c8404691aff6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.229503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2e7f-c8b1-4e8f-8e87-c8404691aff6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.229708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2e7f-c8b1-4e8f-8e87-c8404691aff6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.231363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 448/890] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:05.238080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.238438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.238783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.258270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:05.259563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:05.282236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:05.319229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df747da-bbf3-44fc-9082-627958ad0471 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.341154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 449/891] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:05.368216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.368836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.369135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.369711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.392714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:05.445667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.447140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.555792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 443/892] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:05.561643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.634061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.670954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb79fac-a5e4-404f-8da8-3e20454b134f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.688821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:05.711827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.712788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.713300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.713846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.725224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.727040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.739259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e92f-9ec8-4ad5-8a57-e01d81a4715a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:05.743697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:05.745076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:05.746033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:05.746033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:05.746033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:05.746211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 450/893] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:05.752156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.767280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:05.782189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.783269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.783553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.783981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.784569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.786037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-eb40f276-f534-4eb2-bbfa-3889c9d4c3d9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:05.795005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:05.796012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:05.796869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 444/894] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:05.797816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0f8f-993f-4db9-8a18-fcdfb2af3a96 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.797816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0f8f-993f-4db9-8a18-fcdfb2af3a96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.797816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f0f8f-993f-4db9-8a18-fcdfb2af3a96 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.799572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 445/895] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:05.803928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f0f8f-993f-4db9-8a18-fcdfb2af3a96 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:05.875483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1be6be-a4e7-4ee3-9e93-c981cf1fa39b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.909595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.910196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.910476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.910748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.911380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:05.911677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f0f8f-993f-4db9-8a18-fcdfb2af3a96 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:05.911778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.913061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20551-563d-4a94-b339-68cc6affd066 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.924520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 451/896] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:05.925708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde539c-9312-4874-888d-8d4f4986bbe3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.925985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde539c-9312-4874-888d-8d4f4986bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.926251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde539c-9312-4874-888d-8d4f4986bbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.931722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:05.944932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0fde539c-9312-4874-888d-8d4f4986bbe3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:05.946153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0fde539c-9312-4874-888d-8d4f4986bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:05.946293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:05.946898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.947177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.947447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.948070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:05.948367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:05.949761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389823f3-5d2c-40e9-9a59-14c2cf1bfd99 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:05.954235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 446/897] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:05.955357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78414-bc17-45b7-9747-602790847185 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:05.955637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78414-bc17-45b7-9747-602790847185 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:05.955911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78414-bc17-45b7-9747-602790847185 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:05.957266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 447/898] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:05.957810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78414-bc17-45b7-9747-602790847185 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.008994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.015058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 452/899] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:06.016192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.016480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.016754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.044432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:06.045851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:06.061060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78414-bc17-45b7-9747-602790847185 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.096364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.096987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.097294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.097566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.104250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.105714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.117308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 453/900] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:06.117660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:06.118525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:06.119675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:06.120811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.123756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 448/901] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:06.124579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.172194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.245950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503331a-42a1-4594-beb2-da5208ae0c54 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.259574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b4379-f00d-45f3-b577-c4fed2641c9f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.288368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.289728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.289940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.290299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.290515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.291169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:06.291471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.292954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.298889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.298889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.298889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.298889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.299219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:06.299439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.301983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.307902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 449/902] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:06.308733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 454/903] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:06.309446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.314037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.414747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.417790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc215817-1234-4cef-9f44-caf9d2551b3c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.418741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a23677-6734-4178-ad35-ae11312129bc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.452239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.452778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.452882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.453056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.453356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.453449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.453665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.453934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.454024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.454643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] RBAC: Adding query filter params (role.id=4162e8f52f854c3ea1a74f520a2360ea, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=d253c4509e954c11927f3f1f4563a5ab) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:06.454949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.455352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.456346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.461900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 455/904] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/role_assignments?user.id=d253c4509e954c11927f3f1f4563a5ab&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=4162e8f52f854c3ea1a74f520a2360ea => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:06.462785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.477356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 450/905] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:06 2022] POST /identity/v3/services => generated 241 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:06.477356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.534566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.619977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5d0ae-4e9d-4ca3-a9f1-b76716212dd2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.631357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa963c6-721c-4500-9ef8-a7ddc822011c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.665059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.665713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.665993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.666266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.666972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] RBAC: Adding query filter params (interface=public, service_id=da8fee5567014273aed472f217ed0e61, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:06.667280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.668894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.676146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 456/906] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/endpoints?service_id=da8fee5567014273aed472f217ed0e61&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:06.682545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.684417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.685064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.685373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.685646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.686241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.687631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:06.697091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:06.698097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:06.699010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 451/907] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:06.706050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:06.737986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.805961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3098aa9-8ffa-45ae-91d5-6f3924939824 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.828355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:06.845315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.846009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.846582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.846695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.847359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.848782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.857562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6d53-1e59-4ecc-adf4-24f2309e8aa4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:06.861373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 457/908] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:02:06.898467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:06.899428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:06.899887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:06.900335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:06.901251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:06.901741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:06.903345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:06.909617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 452/909] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:06.916486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:07.033067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a18a3-6bc6-4ae3-8ccb-df8e6dd04097 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:07.069614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:07.070212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:07.070484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:07.070750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:07.071364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:07.071659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:07.073016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:07.078420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 458/910] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:07.085166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:07.179343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:07.197433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f105fb-ddeb-4600-9771-2f356b7f32f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:07.233913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:07.234524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:07.234801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:07.235072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:07.235769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] RBAC: Adding query filter params (interface=public, service_id=e29c08bd3b664b26bd682a390bbdda18, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:07.236075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:07.237475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812446a7-7deb-4d7d-b786-fe52f11c7869 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:07.243051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 453/911] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:02:07 2022] GET /identity/v3/endpoints?service_id=e29c08bd3b664b26bd682a390bbdda18&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:07.249148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:07.301377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:07.354456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5603cf-a9d3-4fd2-85d6-004909a3e6f3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:07.394297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:07.394898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:07.395174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:07.395445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:07.396039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:07.397446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ff1bd-7bea-44c1-be3c-d70985b54ae2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:07.403767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 459/912] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:07 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:02:08.413492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0230a3-0d02-487a-ad4b-77b723759b03 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.414018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0230a3-0d02-487a-ad4b-77b723759b03 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.414018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0230a3-0d02-487a-ad4b-77b723759b03 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.415811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 454/913] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:08.421205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.422050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.422545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.441644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:08.443147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:08.464192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f2a6-c664-4154-99e5-6ebf941ba07a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.464498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f2a6-c664-4154-99e5-6ebf941ba07a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.464744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f2a6-c664-4154-99e5-6ebf941ba07a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.466260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 455/914] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:08.472052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.472333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.472610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.491723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:08.493008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:08.524174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 460/915] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:08.568814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 456/916] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:08.798089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:08.843991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:08.849841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:08.909551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daed1b-ae8b-4cf2-9f94-ebbd854a50b5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:08.926275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:08.946470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:08.947093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.947368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.947630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.950897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7505d3-03ed-4371-bf57-1877de1781df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:08.957788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d46f9be6-1c50-4dec-a7b6-8827190936ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:08.962552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:08.963480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:08.965009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 461/917] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:08.965755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5783-f06c-47c8-bec6-7a2ca400efee None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.965755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5783-f06c-47c8-bec6-7a2ca400efee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.965755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5783-f06c-47c8-bec6-7a2ca400efee None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.967583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 462/918] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:08.969941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5783-f06c-47c8-bec6-7a2ca400efee None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:08.993858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:08.994743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:08.995021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:08.995291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:08.995870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:08.997269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-70a90f7f-6cfa-4841-b0b4-b1b79aee6e78 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:09.006598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:09.007628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.008513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 457/919] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:09.009806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.009806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.009806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.027274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:09.028446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:09.093074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5783-f06c-47c8-bec6-7a2ca400efee None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.094727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.097227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 458/920] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:09.102468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.133059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.133922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.134034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.134291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.134961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.135557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.137647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.150835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 463/921] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:09.151755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.190472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.225951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2afb1-f38b-4ed7-8bd1-259a9f360111 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.256385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8113d74-bc3f-447f-9bba-981dfad5ff78 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.263777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.264695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.265317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.265898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.266733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.267366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.268932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.274868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 459/922] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:09.280221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.292380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.293012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.293399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.293798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.305387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.307064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:09.321819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:09.322898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:09.323802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:09.323802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:09.323802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:09.328552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 464/923] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:09.329549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.353926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.400175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75dfbb-c9f4-434d-bb34-39d16011c126 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.434495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.435104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.435369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.435636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.436261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.436561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.437960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda746ab-81fd-4e55-964a-29b59c2101d6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.440007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.441801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f2b3a-8028-4d2e-8795-f95d62bb5b41 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.443415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 460/924] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:09.444624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528961d-d72b-4e06-8c6a-74b784ae5976 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.445175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528961d-d72b-4e06-8c6a-74b784ae5976 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.445467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528961d-d72b-4e06-8c6a-74b784ae5976 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.449227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 461/925] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:09.449227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528961d-d72b-4e06-8c6a-74b784ae5976 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.483743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.484338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.484610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.484878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.485540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.485843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.487204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.500015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 465/926] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:09.500580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.551289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528961d-d72b-4e06-8c6a-74b784ae5976 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.578056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.584574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.585232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.585511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.585783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.586364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.587736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-85972f7a-785c-4cae-96db-4ac191ef5fb0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:09.596347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:09.597545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:09.598783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 462/927] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:09.600303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.600303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.600303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.605628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe4f39-eea9-4479-aee4-f0bdd93cc911 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.617737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:09.618907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:09.639476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.640081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.640362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.640637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.641363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] RBAC: Adding query filter params (interface=public, service_id=f925e4b8cd8a495286460a679746d01d, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.641672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.643030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.649155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 466/928] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/endpoints?service_id=f925e4b8cd8a495286460a679746d01d&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:09.649300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.679426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.689166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 463/929] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:09.689971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.750601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5aa73-f11d-4a9f-b222-a905def6b151 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.792086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.792701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.792987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.793293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.800620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.802136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.803065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe37ae-b9c0-41ed-90d0-29e16547bd37 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.804693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:09.814360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:09.815340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:09.816302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 467/930] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:09.816889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.853009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.853767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.854045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.854317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.854948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:09.855250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.856735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.865451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 464/931] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:09.866802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.923910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bda041-7a00-4911-a302-574e20595d1e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:09.930023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:09.958312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:09.959487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:09.959773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:09.960037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:09.960631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:09.962008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:09.967842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 468/932] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:02:09.970009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:09.979482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf338b9-fefc-4e80-aca3-b6a07bd60ef8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.020642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.021394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.022215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.022491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.023248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.023552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.025013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.040517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.049955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 465/933] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:10.049955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.092608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afb1f5-706a-40ff-ae73-fa483bd54598 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.141162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.141162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.141162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.141162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.141584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.141584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.141584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.144141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 469/934] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Sep 1 07:02:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:10.144938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.163932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96630d0b-ba53-45f2-a416-de89600be157 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.181628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.200868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.201534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.201817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.202084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.202677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.204047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:10.214971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:10.215928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:10.217186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 466/935] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:10.217985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.257676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519cf29-b8cf-4450-88fb-3bde20d7c614 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.285633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.295694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.296329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.296629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.296902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.307634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.309143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:10.324967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:10.326037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:10.326933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:10.326933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:10.326933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:10.326933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 470/936] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:10.326933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.333329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1efae-f63d-4a12-ba73-08b5a187b139 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.367248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.367774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.368048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.368316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.368971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.370616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.405487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 467/937] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:10.406409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.446536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3c9c2-2295-4408-b0dd-f485343cc90d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.473569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.495370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.496136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.496481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.496821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.497572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.497943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.499503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.514365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 471/938] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:10.515859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.566368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf08b-1383-4ede-92d5-22b6d40c6a23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.572212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.613879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.614510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.614791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.615066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.616692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.617001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.618481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.639508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 468/939] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:10.640780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.650264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875db2c6-6ca5-42e7-8e02-a2a33c74aeeb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.695175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.696658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.697439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.697910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.698996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.699998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.702097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.713308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 472/940] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:10.715035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.744198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.751275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839461f-af19-4055-ab9c-b3a6d36a248a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.788782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.789505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.789821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.790086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.797381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.798837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:10.811670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:10.812606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:10.813522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:10.817660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 469/941] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:10.819133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:10.826804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd830da-6577-4db6-be6b-28f91748ebd2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.829817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:10.863154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.864109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.864580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.865182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.866557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.868105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:10.891798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 473/942] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:02:10.940897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f2c18-4ccb-4db8-a901-f498aa4e544b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:10.987183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:10.987801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:10.988079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:10.988352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:10.989101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:10.989439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:10.991600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e6f48-8c6e-4c32-b037-2cb0495499ee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:11.010687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 470/943] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:11.012114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:11.123048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8dda9-411e-4c08-953b-415ca0892f16 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:11.158623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:11.159554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:11.160008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:11.160453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/d253c4509e954c11927f3f1f4563a5ab/roles/4162e8f52f854c3ea1a74f520a2360ea` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:11.163404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=d253c4509e954c11927f3f1f4563a5ab, role_id=4162e8f52f854c3ea1a74f520a2360ea)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:11.165030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76461e90-bff0-46b3-871a-3162674d47ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:11.184254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 474/944] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:02:11 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/d253c4509e954c11927f3f1f4563a5ab/roles/4162e8f52f854c3ea1a74f520a2360ea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:02:11.919546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad7522-eea3-4550-9d41-577a6cf3585a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:11.920060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad7522-eea3-4550-9d41-577a6cf3585a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:11.920060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad7522-eea3-4550-9d41-577a6cf3585a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:11.921692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 471/945] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:11.926846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:11.928235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:11.928709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:11.950098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:11.951604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:11.966408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:12.021142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 475/946] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:12.291296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad7522-eea3-4550-9d41-577a6cf3585a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:12.308863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:12.405702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad7522-eea3-4550-9d41-577a6cf3585a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:12.441801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:12.442430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.442709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.442982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.443592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:12.444968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8c42b5d6-994d-49ee-956b-bfca2122992c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:12.457379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:12.459358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:12.461046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 472/947] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:12.470736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:12.479381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8b95-3482-4d88-a566-f7330bb0030d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.479666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8b95-3482-4d88-a566-f7330bb0030d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.479947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e8b95-3482-4d88-a566-f7330bb0030d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.481561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 473/948] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:12.486286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572614b-d519-443b-88a6-686e65cec84f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.486570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572614b-d519-443b-88a6-686e65cec84f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.486847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572614b-d519-443b-88a6-686e65cec84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.513099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7572614b-d519-443b-88a6-686e65cec84f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:12.514692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7572614b-d519-443b-88a6-686e65cec84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:12.590685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 474/949] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:12.601691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a9c2c-ffb2-4b37-9961-78bcdaef306a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:12.637465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:12.638401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.638948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.639413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.640234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:12.640728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:12.642359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:12.648170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 476/950] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:12.654201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572614b-d519-443b-88a6-686e65cec84f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:12.664079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:12.776979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572614b-d519-443b-88a6-686e65cec84f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:12.813778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:12.814767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.815246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.815717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.816606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:12.817169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:12.818790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:12.824367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 475/951] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:12.829851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:12.847081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:12.891315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:12.948903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b6696-a9bd-43c0-b113-3459582ac417 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:12.968850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:12.989901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:12.990958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:12.991248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:12.991658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:12.992337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:12.993840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80c3c9-cb1a-484c-a782-5c0b023933a1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.002520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc205a-c702-4ebd-b720-2bcef23f85d0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.022315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 477/952] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:13.022315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc790-eafc-4cfb-a1db-b4be11d0d278 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.022315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc790-eafc-4cfb-a1db-b4be11d0d278 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.022315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abc790-eafc-4cfb-a1db-b4be11d0d278 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.022315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 478/953] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:13.029945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83f-72aa-4d5e-9c1e-6bb4768fa6b4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.030600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83f-72aa-4d5e-9c1e-6bb4768fa6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.031088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276c83f-72aa-4d5e-9c1e-6bb4768fa6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.058379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d276c83f-72aa-4d5e-9c1e-6bb4768fa6b4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:13.060094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d276c83f-72aa-4d5e-9c1e-6bb4768fa6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:13.060232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.060938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.061324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.061720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.062463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.064041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.066871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING keystone.server.flask.common [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 07:02:13.087244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 07:02:13.087244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 07:02:13.087244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [parameters: {'id': 'f475cf0d27394cb6a2b7e5c6ecbd3815', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 07:02:13.087244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f475cf0d27394cb6a2b7e5c6ecbd3815.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f475cf0d27394cb6a2b7e5c6ecbd3815. Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:02:13.095760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 07:02:13.096839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 01 07:02:13.098153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 07:02:13.099319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 07:02:13.100713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:02:13.102433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': 'f475cf0d27394cb6a2b7e5c6ecbd3815', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.104310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:13.106032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f475cf0d27394cb6a2b7e5c6ecbd3815. Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 476/954] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Sep 1 07:02:12 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:13.108528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.157658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:13.165301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 479/955] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:13.166747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb4ac-85bc-4da2-837a-32e2c0b15793 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.167076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb4ac-85bc-4da2-837a-32e2c0b15793 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.167296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccb4ac-85bc-4da2-837a-32e2c0b15793 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.173254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 480/956] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:13.175063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.175350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.175654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.203176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:13.205142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:13.241543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a7d7f-ff36-43f6-a31d-bee36840e4d4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.270129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:13.293819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.294765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.295229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.295682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.304394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.306290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.310694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 481/957] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:13.318255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:13.319200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:13.320264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.322570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 477/958] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:13.328544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.429914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375da35-e607-4a3c-ad1d-008d08427a1a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.442180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.463182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:13.469551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.470165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.470443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.470717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.471521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:13.472225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.474107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.493686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 482/959] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:13.574582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65247dd-7414-419e-842c-e7a4005e0269 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.587876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.617734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.618386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.618662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.618939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.619794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.621220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:13.643036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:13.644179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:13.645166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:13.645943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 478/960] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:13.653058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.722030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:13.731732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b815ad-e3bb-477c-af68-b010eefd2aa7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.779750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.780657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.781100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.782045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.795585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:13.795867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f92911-98e4-4374-95e6-ab96397dd838 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:13.802497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:13.803713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:13.804788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:13.806345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 483/961] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:13.812365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.847228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.848345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.848831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.849547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.850483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:13.850987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.852573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.861443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 479/962] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:13.865783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:13.866613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.921061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37089329-0d24-4f82-8c87-29f807e0f30d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:13.962786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:13.964404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:13.965192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:13.965669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:13.967379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:13.967902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:13.969512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:13.986762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 484/963] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:13.988582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:13.999608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352123c9-4319-4172-b491-4166f5b3a35b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.011256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:14.037960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.038577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.038907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.039179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.039807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:14.040108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.041589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.046898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 480/964] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:14.121926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b9c62-226e-40ed-b163-b960f7ab6bf1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.139370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:14.163549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.164193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.164471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.164741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.176434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.178547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:14.195333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:14.196365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:14.197458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:14.197458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:14.197458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:14.197458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 485/965] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:14.202958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:14.316697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3d202-339c-4665-9d61-f3f0dad54da7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.352277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.352897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.353203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.353479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.354127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:14.354432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.355797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.370152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 481/966] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:14.375511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:14.484472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86486b-e77c-42b7-8b6d-c6a2e9ffe153 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.520932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.521638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.521926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.522202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.530191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.531636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:14.543671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:14.545960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:14.548205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 486/967] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:02:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:14.553333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:14.589349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:14.659338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb771dc-431f-4e1f-a8e2-76c073c8f3e8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.695374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.696263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.696682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.697168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.698047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:14.698504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.700075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.718418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 482/968] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Sep 1 07:02:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:14.726889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:14.754455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:14.834894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ba66-b9eb-4bfd-8383-038a43681a40 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:14.870458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:14.870990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:14.871268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:14.871537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:14.940944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:14.942417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18078a3-38f5-45bc-b813-57ad0215b428 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:14.967445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.050824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 487/969] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:14 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:15.056439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.149826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.158678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b5153-296d-40d0-a561-655ff92cbb3c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.162269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e02bd8-b0d6-494e-93ca-4de41faf5beb None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.162546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e02bd8-b0d6-494e-93ca-4de41faf5beb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.162827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e02bd8-b0d6-494e-93ca-4de41faf5beb None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.164405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 488/970] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:15.169518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.169798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.170073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.190364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:15.191549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:15.199590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649376-5957-4867-84df-2628e717cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.200649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649376-5957-4867-84df-2628e717cfce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.201250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649376-5957-4867-84df-2628e717cfce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.201715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649376-5957-4867-84df-2628e717cfce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.215181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649376-5957-4867-84df-2628e717cfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.217728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649376-5957-4867-84df-2628e717cfce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d9649376-5957-4867-84df-2628e717cfce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:15.238674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:15.239767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:15.241273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:15.241273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:15.241273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:15.243902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 483/971] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:15.249967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649376-5957-4867-84df-2628e717cfce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.285294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 489/972] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:15.361996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649376-5957-4867-84df-2628e717cfce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.396781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.397693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.398154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.398636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.399456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:15.400001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.401578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b7fc5-3a98-433e-8e05-1af2611110e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.414426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 484/973] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:15.420299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.480076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.536684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4440fb-a9bf-4c54-abbd-740f26a67cf0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.572431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.573378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.573825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.574240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.581420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.583016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.589848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2957c44-70ed-4367-a0de-36f937158e03 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.590134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2957c44-70ed-4367-a0de-36f937158e03 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.590422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2957c44-70ed-4367-a0de-36f937158e03 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.595561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 485/974] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:15.596334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2957c44-70ed-4367-a0de-36f937158e03 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8330e2ce-2826-4fed-88c6-452de531d8dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:15.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:15.597431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:15.598340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 490/975] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:15.599207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.599325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.599517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.606776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.620706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:15.621957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:15.697391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 491/976] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:15.697391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.714333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2957c44-70ed-4367-a0de-36f937158e03 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.750608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.758822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.759685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.760292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.760723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.764706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.767925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:15.780350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:15.781662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:15.782673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:15.789836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 486/977] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:15.790977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.812732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e424be-243a-4e34-8477-71ab7ffcd179 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.860952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.861642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.861922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.862247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.863109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:15.863441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.866003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.868737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:15.885198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 492/978] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:15.886855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:15.915638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fc3e3-423c-4457-b1cd-864bb5733ffc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:15.961888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:15.963895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:15.964390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:15.965069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:15.966623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:15.967311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:15.969829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:15.975569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 487/979] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:15.976685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.006764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01223ffb-3733-406f-b470-227f1e975337 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.010823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.051844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.052443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.053296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.053569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.054294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] RBAC: Adding query filter params (role.id=4162e8f52f854c3ea1a74f520a2360ea, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=d253c4509e954c11927f3f1f4563a5ab) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:16.055025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.056829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.064215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 493/980] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:15 2022] GET /identity/v3/role_assignments?user.id=d253c4509e954c11927f3f1f4563a5ab&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=4162e8f52f854c3ea1a74f520a2360ea => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:16.065393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.080577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1310d-74c1-4787-ba9b-e3916b406c23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.112785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.119049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.120050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.120335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.120612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.121303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.122761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.137642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 07:02:16.137642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 07:02:16.137642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [parameters: {'user_id': '0282a4a17da8442d8379977bd69bb396', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 07:02:16.137642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9c06649a-98c1-40a2-b407-00550b1aeea1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:02:16.145007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:02:16.146213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 07:02:16.152526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 07:02:16.154135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 07:02:16.155275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 07:02:16.156356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'user_id': '0282a4a17da8442d8379977bd69bb396', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:16.157494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 07:02:16.158475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 488/981] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Sep 1 07:02:15 2022] POST /identity/v3/users => generated 160 bytes in 170 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b43d-48b7-44e6-aafc-47a4a3a7e8de None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b43d-48b7-44e6-aafc-47a4a3a7e8de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.159508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b43d-48b7-44e6-aafc-47a4a3a7e8de None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.160815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 489/982] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:16.160815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91b43d-48b7-44e6-aafc-47a4a3a7e8de None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.207751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2115901-6e70-414e-8b0e-38e87ec8c683 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.253665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.254532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.254812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.255100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.256236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:16.256637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.257913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8caa7dc-a16d-4aa4-85d4-0a89bcc83f98 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.263039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 494/983] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:16.264226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.264401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.264677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.274885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91b43d-48b7-44e6-aafc-47a4a3a7e8de None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.283855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:16.285021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:16.311104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.313184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.313955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.314225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.314706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.324723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.326416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:16.341444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:16.342465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 07:02:16.343445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:16.344773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 490/984] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:16.345700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.365560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 495/985] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:16.365560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.407588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.463573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04061c-f130-409a-b5e8-c08920806382 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.495409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d71f41-1847-4998-a6ef-9bbb7968b13a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.509245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.509789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.510066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.510341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.511723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] RBAC: Adding query filter params (interface=admin, service_id=f925e4b8cd8a495286460a679746d01d, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:16.512046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.513831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.521160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 491/986] 10.222.0.46 () {56 vars in 1249 bytes} [Thu Sep 1 07:02:16 2022] GET /identity/v3/endpoints?service_id=f925e4b8cd8a495286460a679746d01d&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:16.526441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.533231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.533989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.534329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.534665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.535484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:16.535848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.537647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.578966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 496/987] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Sep 1 07:02:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:16.640699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:16.647105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d9a40-4536-4096-ae76-8eeda263beaa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.667072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.669450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:16.690497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.691119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.691396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.691669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.692709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.694555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307ccff-3acd-4763-8829-4943dc0de1a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.705166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 492/988] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:16 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:02:16.780069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631267b-a3e1-4d30-a1e1-b9528c0525ec None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:16.821610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:16.822403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:16.822774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:16.823141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:16.823914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:16.825501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783a37-5c1e-432a-ba47-4aee7de9027d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:16.845480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 497/989] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:16 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:02:18.032238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b108-3109-414e-af88-57061209764c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.032760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b108-3109-414e-af88-57061209764c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.032840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4b108-3109-414e-af88-57061209764c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.034205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 493/990] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:18.039771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.040146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.040315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.058163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:18.059346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:18.131243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 498/991] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:18.399908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4b108-3109-414e-af88-57061209764c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:18.504972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4b108-3109-414e-af88-57061209764c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:18.539389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:18.540075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.540361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.540635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.541243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:18.542630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-56b5e281-0a65-48ab-80f7-99abd88eba9a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:18.551443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:18.552465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:18.553766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:18.560549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 494/992] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 07:02:18 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:18.560684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:18.567268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:18.650535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc207-c2e9-490b-bf34-9cf359b41314 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.650718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc207-c2e9-490b-bf34-9cf359b41314 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.651105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddc207-c2e9-490b-bf34-9cf359b41314 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.652508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 495/993] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:18.657764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b459-55ba-4c06-ab47-cb2705377f5c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.658050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b459-55ba-4c06-ab47-cb2705377f5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.658325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224b459-55ba-4c06-ab47-cb2705377f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.663551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828eadd-d8d5-4322-87ca-ce53b522e3cf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:18.676389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e224b459-55ba-4c06-ab47-cb2705377f5c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:18.678452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e224b459-55ba-4c06-ab47-cb2705377f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:18.701888 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:18.702499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.702774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.703046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.703670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:18.703969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:18.705367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:18.710289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 499/994] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:18.715727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:18.726951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:18.755107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 496/995] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:18.764038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bc735-1807-4ad7-8de3-62b4f2759bd5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.764321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bc735-1807-4ad7-8de3-62b4f2759bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.764599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bc735-1807-4ad7-8de3-62b4f2759bd5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.766099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 497/996] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:18.771313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.771404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.771689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.790948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:18.792161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:18.797617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:18.831537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9180e-4729-4c48-b387-32a033472053 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:18.862185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 498/997] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:18.867791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:18.868393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:18.868670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:18.868939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:18.869845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:18.870150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:18.871507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:18.876136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 500/998] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:19.029916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.130206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52eb0ee-3944-4a72-a4cf-7e33600b8a1e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.133000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.139284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.170916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.171941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.172466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.173190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.173779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.175268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:19.185595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:19.186608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:19.187602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 499/999] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:19.192774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.237503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9251346-f725-48b6-a71a-c0cb6529ccd5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.270867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.271534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.271813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.272085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.272667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.274062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.275812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:19.284016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:19.285026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:19.285905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:19.290605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 501/1000] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:19.291435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.305522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e707f-19aa-4d97-8a71-b737a6f968c7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.341408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.342021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.342300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.342573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.343199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:19.343502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.344877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.350109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 500/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:19.355710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.392043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.403462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfe9b5-2afb-40c4-881a-0e9b5a8f1de8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.438107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.438712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.438984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.439255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.439869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:19.440166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.441689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.449163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 502/1002] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:19.451989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.467446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6e7b6-af9c-47b0-9bf1-849f0fd4619d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.504988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.505687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.505974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.506243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.507260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:19.507578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.509256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.515360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.515971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 501/1003] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:19.522127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.560867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dec80-3641-441a-9762-6ff88931d01f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.601433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.602041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.602320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.602593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.603226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:19.603528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.604894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.609612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 503/1004] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:19.615321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.631536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.635926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0009c-79cb-4891-895d-80b5005fc7cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.673399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.674469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.674749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.675019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.675731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.677435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.705150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 502/1005] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:02:19.733341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ae3b-76a8-4a0b-873f-0be5405cb101 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.769696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:19.787572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.788197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.788475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.788742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.789484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] RBAC: Adding query filter params (interface=public, service_id=44c1da8e884a4474b7759a00bbc9318c, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:19.789789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.791191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84170936-58f7-49f1-bcfc-4966bf64e5a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.797021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 504/1006] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/endpoints?service_id=44c1da8e884a4474b7759a00bbc9318c&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:19.802610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:19.911545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459e78-7111-437b-895a-c14db648a3cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:19.946099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:19.946737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:19.947102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:19.947384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:19.948013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:19.949429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e770b0-38d6-4bb2-8807-111f21dbe58c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:19.956657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 503/1007] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:19 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:02:20.791222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636c40-c953-4cc7-9f37-9ffd44fd33fc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:20.791737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636c40-c953-4cc7-9f37-9ffd44fd33fc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:20.791813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65636c40-c953-4cc7-9f37-9ffd44fd33fc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:20.793372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 505/1008] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:20.798913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384ea8-ad07-467f-a7e9-05b0a744ed31 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:20.799412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384ea8-ad07-467f-a7e9-05b0a744ed31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:20.799489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384ea8-ad07-467f-a7e9-05b0a744ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:20.818061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-34384ea8-ad07-467f-a7e9-05b0a744ed31 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:20.819776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-34384ea8-ad07-467f-a7e9-05b0a744ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:20.837092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:20.897277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 504/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:21.163327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636c40-c953-4cc7-9f37-9ffd44fd33fc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:21.268514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65636c40-c953-4cc7-9f37-9ffd44fd33fc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:21.304749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:21.305381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:21.305663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:21.305938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:21.306598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:21.307962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e4a98-dbba-4929-b7aa-542990666b0f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:21.333042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 506/1010] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:21 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:21.624001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14a9f-008a-4120-87e0-d8745dc8b165 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:21.624381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14a9f-008a-4120-87e0-d8745dc8b165 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:21.624561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14a9f-008a-4120-87e0-d8745dc8b165 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:21.626207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 505/1011] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:21.631435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8811eed-d88d-4028-b31e-1367dccdc26b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:21.631723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8811eed-d88d-4028-b31e-1367dccdc26b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:21.631996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8811eed-d88d-4028-b31e-1367dccdc26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:21.650609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a8811eed-d88d-4028-b31e-1367dccdc26b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:21.651793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a8811eed-d88d-4028-b31e-1367dccdc26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:21.695697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:21.733186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 507/1012] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:21.874169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c314e-59af-4108-a759-5af639128106 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:21.874728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c314e-59af-4108-a759-5af639128106 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:21.874728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c314e-59af-4108-a759-5af639128106 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:21.876155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 506/1013] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:21.881614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:21.882029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:21.882166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:21.901062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:21.902262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:21.968770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 508/1014] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:21 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:21.994973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c314e-59af-4108-a759-5af639128106 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.095635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c314e-59af-4108-a759-5af639128106 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.128361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.128916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.129226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.129497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.131142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.132573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.134307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 507/1015] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:22.139859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.233266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.240054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6ea77-df31-467d-91b3-8df0c08c4dc1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.241880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:22.256334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:22.282263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.282877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.283155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.283435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.284087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.285777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.325331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 509/1016] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:22 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:22.342403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6e14e-0f3b-4faf-bb69-3934d12f78df None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.380496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.381711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.381989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.382259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.382851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.384263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:22.393867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:22.394965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:22.398251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 508/1017] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:22.401624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.510735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc198c-90ca-4a4e-9527-0cdd50e020df None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.545835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.546443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.546719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.547068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.547698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:22.547998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.549405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.554950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 510/1018] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:22.560585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.645533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:22.665864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf1d3b-1161-406b-b136-16b03c00fcea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.701463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.702072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.702347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.702616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.703238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:22.703538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.704903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca149da-a806-4462-962d-eaf5722a9654 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.709988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 509/1019] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Sep 1 07:02:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:22.715565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:22.735200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:22.827093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087fa3-457c-42f0-b797-be89b2dacb0a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:22.860411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:22.861006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:22.861305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:22.861572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:22.862234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:22.863570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c954594-054b-4d77-91f4-8753759baffb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:22.885379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 511/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:23.268427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06939b-d07d-492c-8bb3-d7f1cd5250c4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:23.268952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06939b-d07d-492c-8bb3-d7f1cd5250c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:23.268952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06939b-d07d-492c-8bb3-d7f1cd5250c4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:23.270499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 510/1021] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:23.275946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:23.276240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:23.276519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:23.295171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:23.296368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:23.366825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:23.374350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 512/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:23.644394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06939b-d07d-492c-8bb3-d7f1cd5250c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:23.745355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06939b-d07d-492c-8bb3-d7f1cd5250c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:23.783935 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:23.784621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:23.784905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:23.785202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:23.785861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:23.787223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:23.796112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:23.801276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:23.802230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 511/1023] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 07:02:23 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:23.803814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:23.904078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9705a-14b9-4459-bd2f-98d8152a560e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:23.938357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:23.938974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:23.939256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:23.939531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:23.940165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:23.940468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:23.941864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:23.953299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 513/1024] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:23.956741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:23.972769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:24.062467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f808291-73d0-4a76-90e8-fab228625418 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:24.096591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:24.097208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.097483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.097747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.098364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:24.098660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:24.100038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:24.104934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 512/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:24.110571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:24.172235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:24.211492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108f9c0-d278-481c-8ce7-e349e12f655f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:24.244984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f966-accd-4686-8307-e68196005439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:24.245651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f966-accd-4686-8307-e68196005439 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.245930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f966-accd-4686-8307-e68196005439 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.246198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f966-accd-4686-8307-e68196005439 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.246934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f966-accd-4686-8307-e68196005439 None admin] RBAC: Adding query filter params (interface=public, service_id=f57cb2674c624b9faac1a42cf9ce8e47, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:24.247206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f966-accd-4686-8307-e68196005439 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:24.248561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f966-accd-4686-8307-e68196005439 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:24.253769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 514/1026] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:02:24 2022] GET /identity/v3/endpoints?service_id=f57cb2674c624b9faac1a42cf9ce8e47&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:24.259155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:24.287581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5b88c-ce22-4517-b72d-3304c848808e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.287877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5b88c-ce22-4517-b72d-3304c848808e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.288154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5b88c-ce22-4517-b72d-3304c848808e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.289744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 515/1027] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:24.294454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.294738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.295015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.313562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:24.314752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:24.341950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:24.371691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e4e16-43f0-4154-b90c-86a7853bc4bb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:24.384341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 516/1028] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:24.407532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:24.408140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.408414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.408684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.409313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:24.410669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:24.417837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 513/1029] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:02:24 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:02:24.656483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:24.715262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:24.764278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b070cd-be35-4b19-abe2-0fa5f2ac4cd0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:24.800415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:24.801055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.801358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.801634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.802799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:24.804238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f119a-1647-465d-a2eb-16ec948e78dc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:24.805995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 517/1030] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:24.811520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:24.873810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e696f-6a71-4e24-9ce8-06b3ac38bcc5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.874135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e696f-6a71-4e24-9ce8-06b3ac38bcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.874362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e696f-6a71-4e24-9ce8-06b3ac38bcc5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.875687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 518/1031] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:24.880450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.880769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.881006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.902932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:24.904141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:24.918457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561a700-dada-42c3-b879-f9d2a7c4bf1c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:24.954160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:24.954794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:24.955072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:24.955340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:24.955991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:24.957447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:24.970583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:24.978614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 519/1032] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:24.984351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 514/1033] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:02:24 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:25.253020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:25.310163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:25.354561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de05060-fdb7-4000-ba18-6d62127a9b52 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:25.392012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:25.392648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:25.392929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:25.393230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:25.403609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:25.408417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:25.409377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:25.410224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 520/1034] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:25.415359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:25.522715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f55278-b029-43ab-9fba-86e703a1f1d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:25.556040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:25.556864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:25.557341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:25.557772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:25.558583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:25.559092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:25.560598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:25.573626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 515/1035] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:25.575140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:25.680653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0093c30b-6c55-4a99-a70f-c475dd944a82 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:25.696597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:25.714837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:25.715451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:25.715724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:25.715994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:25.726471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:25.727896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:25.741981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:25.742926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:25.743944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:25.743944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:25.743944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:25.743944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 521/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:25.750458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:25.810239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:25.853421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07f451-0bdb-407a-84e5-1b1b0456a5c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:25.887666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:25.888272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:25.888548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:25.888816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:25.889481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:25.889784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:25.891145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:25.905262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 516/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:25.909510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.011295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670339-7472-4bd0-9d80-795db7acfa44 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.046468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.047181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.047464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.047739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.055335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.056764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:26.067734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:26.069219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:26.075712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 522/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:02:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:26.077694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.180620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918e7e5-de78-44ed-ae93-1c5d4d8baa25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.198697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.216570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.217197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.217479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.217747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.218522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:26.218823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.220288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f9dd6-de09-4fca-9f05-f0dc12390681 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.238388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 517/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:26.244423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.274820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.336507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f571-da38-4173-ac92-fde31da17a93 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.336785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f571-da38-4173-ac92-fde31da17a93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.336977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f571-da38-4173-ac92-fde31da17a93 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.338465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 518/1040] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:26.342962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.343244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.343517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.349128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a042f0-25c0-407e-a92c-ba132433dd4c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.361895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:26.363074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:26.385328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.386363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.386642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.386906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.433288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 519/1041] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:26.460623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.462122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.490103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.567595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 523/1042] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:26.569914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.676142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb5f7-c572-4d27-b3aa-c7d805e3aa38 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.683479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.704288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.711846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.712481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.712757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.713026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.725021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.726810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:26.744293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:26.750238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:26.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:26.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:26.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:26.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 520/1043] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:26.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.819280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e63ae-e09e-41b7-a8cb-23618de89a59 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.848556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.857462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.858110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.858383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.858652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] PATH_INFO: `/v3/services/image` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.859228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.859879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d12d86-d60b-4583-a3d6-e7055dbd55e0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:26.860636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:26.870772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:26.871782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:26.872644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 524/1044] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:26.877700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.900114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:26.900779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:26.901061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:26.901359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:26.901995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:26.902299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:26.903669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:26.916347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 521/1045] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:26.922054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:26.942308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:26.982962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31be4c-a715-4261-a620-cbf3cf8e3e5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.016621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.017271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.017527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.017799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.018421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:27.018720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.020084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8192e-4b59-421f-aee3-804d59c4ee51 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.024634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567a02f-7918-4432-b5ac-b5a0f9370a70 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.025057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 525/1046] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:27.026105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1ec00-b215-433c-bc4b-cf97cbaab5ea None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.026384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1ec00-b215-433c-bc4b-cf97cbaab5ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.026654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1ec00-b215-433c-bc4b-cf97cbaab5ea None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.027830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 526/1047] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:27.030925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1ec00-b215-433c-bc4b-cf97cbaab5ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.063215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.063847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.064127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.064399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.071641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.073083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4b31870e-5ec7-4aff-93ba-71a8cbd96024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:27.084093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:27.085034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:27.086023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 522/1048] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:27.086522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.086765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.087680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.090853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.105494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:27.106686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:27.137510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1ec00-b215-433c-bc4b-cf97cbaab5ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.170576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.171227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.171506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.171803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.172444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] RBAC: Adding query filter params (type=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:27.172746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.172978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 523/1049] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:27.173836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.174142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.186210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 527/1050] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:27.187389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.187957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.288300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3aa66-07fa-4e1a-bbd5-26a6d52567a3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.291619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca677f71-c699-4e00-9139-6ea886c6c19f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.314302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.323432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.324036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.324310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.324577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.325287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:27.325591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.327013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.331312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.331910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.332188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.332462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.333138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.335111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.338759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 524/1051] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:27.344463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.365530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 528/1052] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:02:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:02:27.437208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.442780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.475321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6383-a177-43a9-b9dd-5e8a57705f3e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.521681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.522669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.522956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.523286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.524014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] RBAC: Adding query filter params (role.id=4162e8f52f854c3ea1a74f520a2360ea, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:27.524323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.526186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.531700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 525/1053] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=4162e8f52f854c3ea1a74f520a2360ea => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:27.561054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228488e-2b9c-4e45-bd3f-a0a3059fd784 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.599901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.600886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.601372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.601824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.614557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:27.622552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:27.624152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:27.625412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:27.628510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 529/1054] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:27.634591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.739941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f5716-d435-4534-b6dc-4466bfd8917a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.772106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.781208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.781819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.782095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.782362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.783041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:27.783340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.784702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.794389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 526/1055] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:27.800091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:27.838511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:27.907002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466706d-a88f-4ba1-ba75-4fe88ca8a3a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:27.941317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:27.941950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:27.942230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:27.942503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] PATH_INFO: `/v3/users/f522430d2c8e47d4b2f8611e55d2945d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:27.955608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] RBAC: Authorizing `identity:get_user(user_id=f522430d2c8e47d4b2f8611e55d2945d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:27.957052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:27.963881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 530/1056] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/users/f522430d2c8e47d4b2f8611e55d2945d => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:27.965572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:28.065561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bedfa-c09d-4131-a0f7-917938ba8e08 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:28.099973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:28.100535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:28.100814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:28.101088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:28.158613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:28.170105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:28.171565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:28.270679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 527/1057] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 07:02:28.277418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:28.374571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:28.381999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187b7bc-fb3b-4bb3-8028-6227ac0d7628 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:28.417422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:28.418047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:28.418324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:28.418594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:28.419709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:28.421138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df9e8e-087b-4e59-9dcf-316c58eefd54 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:28.422932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 531/1058] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:28 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:28.428784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:28.522227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:28.533425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747d5f-ce03-4510-ba44-d9c7cdecfddf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:28.568712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:28.569577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:28.570026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:28.570448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:28.571320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f, user.id=f522430d2c8e47d4b2f8611e55d2945d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:28.571793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:28.573400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ef1ac-aaab-483d-88b5-72a345f55d63 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:28.579537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 528/1059] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:28 2022] GET /identity/v3/role_assignments?user.id=f522430d2c8e47d4b2f8611e55d2945d&scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:29.313975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68e20d-442f-4678-a537-03ba686c505f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.314533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68e20d-442f-4678-a537-03ba686c505f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.314533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68e20d-442f-4678-a537-03ba686c505f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.316017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 532/1060] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:29.322691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905cbc-d1ae-4234-b9d2-7f7f2f351edc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.322983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905cbc-d1ae-4234-b9d2-7f7f2f351edc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.323258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905cbc-d1ae-4234-b9d2-7f7f2f351edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.342788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13905cbc-d1ae-4234-b9d2-7f7f2f351edc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:29.343985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13905cbc-d1ae-4234-b9d2-7f7f2f351edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:29.411772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 529/1061] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:29.428408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4f4c0-71e5-4f4e-9acb-bde9eff1ec8b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.428684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4f4c0-71e5-4f4e-9acb-bde9eff1ec8b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.428959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4f4c0-71e5-4f4e-9acb-bde9eff1ec8b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.430421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 533/1062] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:29.435234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.435525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.435794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.454173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:29.455342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:29.520961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 530/1063] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:29.679579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4f4c0-71e5-4f4e-9acb-bde9eff1ec8b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:29.787581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4f4c0-71e5-4f4e-9acb-bde9eff1ec8b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:29.789154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:29.804402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:29.825173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:29.825780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.826056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.826327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.857733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:29.892899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed573755-ef11-4fc6-b2f2-1fa2dc87c283 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:29.899063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:29.900632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:29.927860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:29.928641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:29.928931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:29.929232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:29.939744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:29.945197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:29.946129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:29.947162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 531/1064] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:29.952182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.013222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 534/1065] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:30.014990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.030555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.055972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5663864-191b-4a4f-b799-5591d7e1c232 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.089286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.091295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.092000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.092293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.092682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.093379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:30.093682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.095066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.103648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 532/1066] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:30.108977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.134280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e15116-1df9-499f-8b23-3ba4442e2c24 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.177166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.178337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.178612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.179044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.190982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.192355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:30.208884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:30.210045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:30.210978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:30.210978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.210978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:30.210978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 535/1067] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:30.216881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb77c9b-d203-4c13-8252-0d0d5eaba239 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.218276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.253825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.254704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.254976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.255249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.258307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.267809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.269317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:30.285015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:30.286186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:30.287640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:30.287640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.287640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:30.287640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 533/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:30.292194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.336676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f64a0-17ee-4409-8b95-6d3a2d509add None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.351983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.373729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.374336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.374611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.374881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.375518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:30.375820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.377218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb09ef-8c9d-46c0-b12b-4a49ae1790a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.390670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 536/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:30.409072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0aaaf-c745-4381-a81c-deddbb306704 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.447912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.448455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.448734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.449010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.449700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:30.450004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.451402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.461706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f2e6-854f-4209-ab00-b44055da91f2 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.461982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f2e6-854f-4209-ab00-b44055da91f2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.462248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1f2e6-854f-4209-ab00-b44055da91f2 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.463750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 537/1070] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:30.465549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 534/1071] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:30.468966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.469282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.469562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.470836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.490911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:30.492167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:30.525696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.571123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 538/1072] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:30.584927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c611a-266a-4b4f-ad24-fd952a28ecfe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.623550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.624288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.624577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.624845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.633218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.634680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:30.647364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:30.648341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:30.649401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 535/1073] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:30.656039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.761521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda35ce-1f82-4bc4-b6f1-1a7efa3863f5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.766256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.798054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.798653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.798925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.799191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.799903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:30.800206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.801649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.818599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 539/1074] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:30.824162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.848485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.922239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d57dc6-40e7-45c7-851c-71c86558f358 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.930945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:30.945836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9f73d-72f5-4dcd-af54-205ecf09e845 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:30.956203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.956832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.957148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.957419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.969092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:30.970581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:30.980415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:30.981030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:30.981338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:30.981612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:30.985666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:30.986746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:30.987917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:30.987917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:30.987917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:30.987917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 536/1075] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:30.992721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:30.996544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:31.001348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:31.002434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:31.003646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 540/1076] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:31.008649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.033728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.097067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea578ee-2562-44ee-b062-545f4e4a173f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.112089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be0618-b04d-4563-a2e3-2e66c15f449e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.129587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.130187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.130977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.131254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.131890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:31.132197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.133609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.145046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 537/1077] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:31.150267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.150890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.151160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.151424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.151830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.152086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:31.152385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.153780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.162483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 541/1078] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:31.167981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.173890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.250590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.251480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c8b44-5110-422f-804e-0b7b6a9ac520 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.267511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9a45c-c362-418d-b824-078236b022c5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.288288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.288853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.289157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.289432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.295945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.297423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.299986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.300596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.300870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.301166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] PATH_INFO: `/v3/users/f522430d2c8e47d4b2f8611e55d2945d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.302261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] RBAC: Authorizing `identity:get_user(user_id=f522430d2c8e47d4b2f8611e55d2945d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.303689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.305268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 542/1079] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:31 2022] GET /identity/v3/users/f522430d2c8e47d4b2f8611e55d2945d => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:31.308570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:31.309948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:31.310863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 538/1080] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:31.315469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.315594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.395512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.413533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6b273-434b-42fd-a8a5-60d6eb923b41 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.414043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd28b-94e0-4a90-ad44-0d1d00489d51 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.446252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.446889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.447160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.447341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.447481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.447940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.448219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.448494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.448592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.449206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:31.449512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.449972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.450927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.451574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 543/1081] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:31 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:31.457360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.467483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 539/1082] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:31.468517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:31.480150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.557647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d033a8-56bd-4763-84d2-dfb6fc4f118b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.566018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352a65e-8f4f-4ad0-89a7-5bc7cf0f8574 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:31.590901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.591540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.591819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.592094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f/users/f522430d2c8e47d4b2f8611e55d2945d/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.593991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3bb17a009a94d88a1e6f7a12e9a680f, user_id=f522430d2c8e47d4b2f8611e55d2945d, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.595400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdab20-4585-4e7d-8ed5-c5751ec09530 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.599849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:31.600590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:31.600966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:31.601359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/4162e8f52f854c3ea1a74f520a2360ea` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:31.603784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=3d44b8c931e047408a2d9d4baa490806, role_id=4162e8f52f854c3ea1a74f520a2360ea)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:31.605242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e542dc-f625-46e5-bc30-6875cf180218 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:31.607729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 544/1083] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:02:31 2022] PUT /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f/users/f522430d2c8e47d4b2f8611e55d2945d/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:02:31.613569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:31.620151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 540/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:02:31 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/3d44b8c931e047408a2d9d4baa490806/roles/4162e8f52f854c3ea1a74f520a2360ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:02:33.496294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a091-0057-4afb-b2b1-ae39b64a1d0e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:33.496829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a091-0057-4afb-b2b1-ae39b64a1d0e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:33.496829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a091-0057-4afb-b2b1-ae39b64a1d0e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:33.498301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 545/1085] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:33.503537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:33.503944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:33.504088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:33.508770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a273e-1fbb-4986-9654-bfa669abb9ad None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:33.509052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a273e-1fbb-4986-9654-bfa669abb9ad None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:33.509354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a273e-1fbb-4986-9654-bfa669abb9ad None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:33.510598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 546/1086] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:02:33.515401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:33.515679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:33.515952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:33.523084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:33.524277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:33.533837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:33.534982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:33.590121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 541/1087] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:33.601663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 547/1088] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:33.858879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:33.873792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:33.884948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:33.956225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c3872-6432-4b49-a3ac-864d412ccc1e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:33.971955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ff453-13d2-405f-b32c-daa79ee07a2e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:33.985163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:33.991406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:33.992026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:33.992297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:33.992569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.003714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:34.006232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.006848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.007124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.007391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:34.008970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:34.010168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:34.011362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 542/1089] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:34.015946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.020665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:34.026097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:34.027271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 07:02:34.028450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:34.032158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 548/1090] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Sep 1 07:02:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:34.032805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.099653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.114647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af38c24-b795-4986-9cb7-b01f852f763b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.131021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0ad-4627-40cf-9c8f-ec10c2711a47 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.149304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.149910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.150188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.150462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.151117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:34.151417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.152786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.160665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 543/1091] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.164203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.164803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.165081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.165665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.166323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:34.166424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.166627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.167978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.180657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 549/1092] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:34.181635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.221310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.268495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd8ad5-bb75-4d8d-a685-30b94a0c1b7d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.282935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539356f6-a9c8-43ee-a924-fb5ddb694576 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.301751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.302447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.302731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.303007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] PATH_INFO: `/v3/users/f522430d2c8e47d4b2f8611e55d2945d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.304143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] RBAC: Authorizing `identity:get_user(user_id=f522430d2c8e47d4b2f8611e55d2945d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.305630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.307222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 544/1093] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/users/f522430d2c8e47d4b2f8611e55d2945d => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.310865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.313051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.317660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.318859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.319109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.319386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.330065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.331578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:34.345930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:34.346920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:34.348329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:34.348329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:34.348329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:34.348329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 550/1094] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:34.352855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.416749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8eee3-6d6a-43d0-abdd-a608913a4ffe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.436611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.452422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.453031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.453345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.453650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.459206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880084c-7d18-4a87-a38f-e4299fb032fd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.492233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.492885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.493199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.493480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.494117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:34.494420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.496063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.508701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 551/1095] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.514284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.520447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.523032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.559023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.613178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e2793-ded2-489b-9552-dcd704ac4d41 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.619543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 545/1096] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 07:02:34.625617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.645013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.647065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.647826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.648113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.648383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.655613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.657050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:02:34.669581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:34.670673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:34.671831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 552/1097] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:02:34.676337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.731007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ed22e-1ce3-4d96-87b3-173965ab39e9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.767968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.768602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.768877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.769173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.770288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.772385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.773858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5a3c9-edb8-4f7a-9ade-3b5638ca4e01 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.774302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 546/1098] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.776162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.780380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.807744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.808353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.808631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.808903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.809659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:34.809966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.811384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.828573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 553/1099] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.834769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:34.867402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:34.880886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ad7eb-b785-42c5-91b3-2ee151e4b4ad None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.914749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.915344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.915623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.915896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:34.916620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f, user.id=f522430d2c8e47d4b2f8611e55d2945d) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:34.916926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:34.918389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:34.924055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 547/1100] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/role_assignments?user.id=f522430d2c8e47d4b2f8611e55d2945d&scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:02:34.933271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358d1bf-06e8-4add-a961-b6eb699bfabc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:34.971553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:34.972179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:34.972457 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:34.972729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.006592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:35.046423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.047978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.147764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 554/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 07:02:35.153496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:35.251758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d3c4a-7de9-4643-b103-2b476956eb52 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:35.284517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:35.285163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:35.285448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:35.285724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.296082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.297620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:02:35.311908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:35.313048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:02:35.314201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:02:35.314201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:02:35.314201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:35.317269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 548/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:35.319355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:35.378523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:35.416532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1af8-8815-4aaa-87be-d6e1646f754d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:35.458663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:35.459545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:35.459825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:35.460092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.460732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:35.461029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.462419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.473528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 555/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:35.479785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:35.483543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:35.578572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be37c88-79b5-4d3c-90c1-ef22e1cc1bab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:35.611463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:35.612157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:35.612440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:35.612711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.620069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.621549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:02:35.632617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:35.633709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:02:35.634785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:02:35.637253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 549/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:35.639497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:35.741303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f7a1-b088-420d-80bd-828eca95714a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:35.767940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:35.779773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:35.780376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:35.780654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:35.780926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.781646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:35.781947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.783370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114a93a-6bad-4a43-bc6a-93543c03612a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.795453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 556/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:35.801594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:35.863139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:35.901624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fedc968-b30a-494c-a719-b7b4a1126723 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:35.934506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:35.935105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:35.935384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:35.935655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:35.936382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] RBAC: Adding query filter params (role.id=4162e8f52f854c3ea1a74f520a2360ea, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=3d44b8c931e047408a2d9d4baa490806) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:35.936690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:35.938158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b16f1-fa39-451d-b5f0-3436942ef273 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:35.943834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 550/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:35 2022] GET /identity/v3/role_assignments?user.id=3d44b8c931e047408a2d9d4baa490806&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=4162e8f52f854c3ea1a74f520a2360ea => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:36.847924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e3e5a-9ec8-4f29-8d8e-d9ff8a4c5cbd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:36.848461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e3e5a-9ec8-4f29-8d8e-d9ff8a4c5cbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:36.848461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e3e5a-9ec8-4f29-8d8e-d9ff8a4c5cbd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:36.849834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 557/1107] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:36.854790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:36.855167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:36.855341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:36.873810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:36.874974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:36.938096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 551/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:37.201480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e3e5a-9ec8-4f29-8d8e-d9ff8a4c5cbd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:37.298787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e3e5a-9ec8-4f29-8d8e-d9ff8a4c5cbd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:37.330962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:37.331566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:37.331841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:37.332110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:37.333222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:37.334689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8586d8-ef38-44e3-bc07-e0411b744347 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:37.336216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 558/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:37.341885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:37.398755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:37.438605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d342c77-9110-4506-8a36-87494d2178fb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:37.471398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:37.471936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:37.472216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:37.472486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:37.473142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:37.475198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f0ac6-2233-4dcc-bc80-60dad510616c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:37.495518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 552/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:02:37 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:39.405512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c92693-be15-4ae8-8702-8a2a845ef200 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:39.406065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c92693-be15-4ae8-8702-8a2a845ef200 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:39.406065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c92693-be15-4ae8-8702-8a2a845ef200 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:39.407416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 559/1111] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:39.412920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:39.413320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:39.413495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:39.432026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:39.433238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:39.497320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 553/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:39.767948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c92693-be15-4ae8-8702-8a2a845ef200 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:39.817473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:39.866404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c92693-be15-4ae8-8702-8a2a845ef200 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:39.898971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:39.899572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:39.899846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:39.900113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] PATH_INFO: `/v3/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:39.901485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] RBAC: Authorizing `identity:get_role(role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:39.902831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:39.904099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 560/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:39 2022] GET /identity/v3/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:39.909739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:39.953355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:40.007242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc04cdb-9651-49c3-8761-75b823a538a4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:40.040216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:40.040831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:40.041108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:40.041409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] PATH_INFO: `/v3/users/af217587494d44218ad19c25a4a0f2e5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:40.054144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] RBAC: Authorizing `identity:get_user(user_id=af217587494d44218ad19c25a4a0f2e5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:40.055570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:40.057176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 554/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:39 2022] GET /identity/v3/users/af217587494d44218ad19c25a4a0f2e5 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:40.062699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:40.157982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fca453-2bba-4770-8a93-f6a2e06f58b1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:40.173790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:40.190159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:40.190748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:40.191022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:40.191289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:40.256129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:40.257626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:40.352140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 561/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:40.357923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:40.453755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe59530-5294-40bf-acc5-9853bc5f1179 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:40.455221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:40.487208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:40.487815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:40.488089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:40.488355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:40.489473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:40.491407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbbf2c-7664-4d1d-9fd0-8fba55a54226 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:40.493024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 555/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:40 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:40.499155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:40.516984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:40.592733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49af8c5-a3b4-42cc-8f93-5d13826847e8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:40.625568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:40.626087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:40.626360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:40.626627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:40.627334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f, user.id=af217587494d44218ad19c25a4a0f2e5) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:40.627634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:40.629039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cb1d4-3ec7-4d7a-b51f-5f4d8e8e96f1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:40.634721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 562/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:40 2022] GET /identity/v3/role_assignments?user.id=af217587494d44218ad19c25a4a0f2e5&scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:42.488572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc8c3-d572-4f51-bd05-1aa391dc156c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:42.489099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc8c3-d572-4f51-bd05-1aa391dc156c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:42.489099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfc8c3-d572-4f51-bd05-1aa391dc156c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:42.491047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 556/1118] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:42.496391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:42.496792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:42.496941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:42.514604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:42.515796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:42.579891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 563/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:42.846785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfc8c3-d572-4f51-bd05-1aa391dc156c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:42.945045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfc8c3-d572-4f51-bd05-1aa391dc156c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:42.978111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:42.978726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:42.979002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:42.979275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] PATH_INFO: `/v3/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:42.980650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] RBAC: Authorizing `identity:get_role(role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:42.982054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:42.983348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 557/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:42 2022] GET /identity/v3/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:42.989063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:43.002993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:43.085503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0c653-6adb-44c3-b320-31d11584467a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:43.117793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:43.118406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:43.118683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:43.118954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] PATH_INFO: `/v3/users/af217587494d44218ad19c25a4a0f2e5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:43.120311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] RBAC: Authorizing `identity:get_user(user_id=af217587494d44218ad19c25a4a0f2e5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:43.121762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:43.123298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 564/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:42 2022] GET /identity/v3/users/af217587494d44218ad19c25a4a0f2e5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:43.128845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:43.167636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:43.224332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40f17f-78d6-4c57-b011-be0c776109aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:43.257106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:43.257742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:43.258019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:43.258288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:43.259393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:43.260817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd23053-ce8e-4fcc-aa1c-05ad64f3017a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:43.262444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 558/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:43 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:43.268161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:43.345971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:43.365490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd451e-4a63-4f45-a316-ce63f34ae28e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:43.397592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:43.398225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:43.398500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:43.398769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f/users/af217587494d44218ad19c25a4a0f2e5/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:43.400829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3bb17a009a94d88a1e6f7a12e9a680f, user_id=af217587494d44218ad19c25a4a0f2e5, role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:43.402269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebc076-9dc1-4478-9c9c-0b3d16f43251 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:43.413902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 565/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:02:43 2022] PUT /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f/users/af217587494d44218ad19c25a4a0f2e5/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:02:45.291968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dff2db-38ee-48c9-b924-6cdb191346b3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:45.292558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dff2db-38ee-48c9-b924-6cdb191346b3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:45.292558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dff2db-38ee-48c9-b924-6cdb191346b3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:45.293908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 559/1124] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:45.298901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:45.299320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:45.299467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:45.317897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:45.319081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:45.389014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 566/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:45.685625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dff2db-38ee-48c9-b924-6cdb191346b3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:45.792770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dff2db-38ee-48c9-b924-6cdb191346b3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:45.800999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:45.829589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:45.830213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:45.830487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:45.830752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] PATH_INFO: `/v3/roles/7e7f959c5705479f8adf2ebf79601bd1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:45.832116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] RBAC: Authorizing `identity:get_role(role_id=7e7f959c5705479f8adf2ebf79601bd1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:45.833573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:45.835020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 560/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:45 2022] GET /identity/v3/roles/7e7f959c5705479f8adf2ebf79601bd1 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:45.841605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:45.912619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:45.945768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf932e-d084-4c5b-afe0-7dbc15729e61 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:45.978439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:45.979107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:45.979379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:45.979644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] PATH_INFO: `/v3/users/af217587494d44218ad19c25a4a0f2e5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:45.980759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] RBAC: Authorizing `identity:get_user(user_id=af217587494d44218ad19c25a4a0f2e5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:45.982567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:45.984140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 567/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:45 2022] GET /identity/v3/users/af217587494d44218ad19c25a4a0f2e5 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:45.990624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:46.088325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf67ed-dfdd-49ef-82e4-6d8de971d89d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:46.125048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:46.125687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:46.125966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:46.126270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:46.167882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:46.192543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:46.194057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:46.287795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 561/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:46.293689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:46.388999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54579fcc-d8a1-4e8f-923f-cd8bde5a8671 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:46.423286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:46.423903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:46.424258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:46.424748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] PATH_INFO: `/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:46.426716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=c3bb17a009a94d88a1e6f7a12e9a680f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:46.428473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d557-d52b-42cf-bcdb-88a6306e7dd9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:46.430217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 568/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:46 2022] GET /identity/v3/projects/c3bb17a009a94d88a1e6f7a12e9a680f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:46.436360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:46.507933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:46.532109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deac63b-94b1-4e0e-8cf3-beaf2a2a5994 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:46.564663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:46.565219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:46.565500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:46.565769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:46.566479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] RBAC: Adding query filter params (role.id=7e7f959c5705479f8adf2ebf79601bd1, scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f, user.id=af217587494d44218ad19c25a4a0f2e5) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:46.566781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:46.568225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc899b-1991-410f-b09c-74d45b22f73c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:46.573762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 562/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:46 2022] GET /identity/v3/role_assignments?user.id=af217587494d44218ad19c25a4a0f2e5&scope.project.id=c3bb17a009a94d88a1e6f7a12e9a680f&role.id=7e7f959c5705479f8adf2ebf79601bd1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:48.454980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb884b-ddbf-4368-86a7-fac1c5425ebf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:48.455526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb884b-ddbf-4368-86a7-fac1c5425ebf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:48.455526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb884b-ddbf-4368-86a7-fac1c5425ebf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:48.456829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 569/1131] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:48.462456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:48.462874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:48.463008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:48.480649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:48.481863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:48.545895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 563/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:48.815948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb884b-ddbf-4368-86a7-fac1c5425ebf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:48.825953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:48.913903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb884b-ddbf-4368-86a7-fac1c5425ebf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:48.946976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:48.947579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:48.947854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:48.948122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:48.949211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:48.950620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35f4bf-1822-4662-8191-12f36463e20a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:48.952134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 570/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:48.957892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:49.021737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:49.054742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808f44f-eec5-4543-8977-ec9c48bef088 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:49.086762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:49.087361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:49.087639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:49.087913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:49.088558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:49.090015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a619efb-750c-47b2-a77e-3438449da308 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:49.116956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 564/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:02:48 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:51.039317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16494e-5789-414f-a3ef-4c6768a6964a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:51.039860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16494e-5789-414f-a3ef-4c6768a6964a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:51.039860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16494e-5789-414f-a3ef-4c6768a6964a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:51.041216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 571/1135] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:51.046256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:51.046668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:51.046803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:51.064454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:51.065676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:51.133852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 565/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:51.400141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16494e-5789-414f-a3ef-4c6768a6964a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:51.472685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:51.501931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16494e-5789-414f-a3ef-4c6768a6964a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:51.534806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:51.535415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:51.535691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:51.535961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:51.537064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:51.538499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503bcc1-ef3b-40d2-bc44-4a87e7c29e1b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:51.540033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 572/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:51.545807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:51.593090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:51.641425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed367fbd-fab6-46b7-85f5-626ac791ee1b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:51.672984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:51.673550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:51.673823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:51.674088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:51.674720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:51.676140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27260d-710b-4946-8d5c-ff59b39e9120 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:51.696814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 566/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:02:51 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:02:53.629154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93846752-1d96-41c4-8e33-4549b59db061 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:53.629717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93846752-1d96-41c4-8e33-4549b59db061 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:53.629717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93846752-1d96-41c4-8e33-4549b59db061 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:53.631008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 573/1139] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:53.636224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:53.636606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:53.636780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:53.654584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:53.655768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:53.720019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 567/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:02:53.988054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93846752-1d96-41c4-8e33-4549b59db061 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.085535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93846752-1d96-41c4-8e33-4549b59db061 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:54.117812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:54.118412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:54.118689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:54.118960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:54.128831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:54.133647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:54.134800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:54.136063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 574/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:54.140528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.174136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:54.238925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d8f14-8474-4d43-971c-a04f6e5dc728 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:54.271834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:54.272424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:54.272697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:54.272962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:54.273639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:54.273936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:54.275295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:54.283002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 568/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:54.288786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.292867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:54.386018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857278d-4eaa-4f6a-a7b5-41cc1094da2b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:54.418166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:54.418766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:54.419042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:54.419313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] PATH_INFO: `/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:54.431619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] RBAC: Authorizing `identity:get_user(user_id=7e5d31d4baf34cddb93a2f0adfef85d2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:54.433053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:54.434622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 575/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:54 2022] GET /identity/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:54.440481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.535483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:54.539592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c060c71-7e38-4ba3-8606-6363cd28af79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:54.571758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:54.572351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:54.572632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:54.572902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:54.637110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:54.638564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:54.728243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:54.733661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 569/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:02:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:02:54.738726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.799401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:54.839246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c0ea9-c4d4-456e-b64d-ec8b99667bee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:54.874985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:54.875624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:54.875903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:54.876178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] PATH_INFO: `/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:54.877309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] RBAC: Authorizing `identity:get_project(project_id=7cbb048dcc364049b9a2a1a4fa18ce6f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:54.878740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41039f2a-928c-4418-a112-9dad108b7022 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:54.881007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 576/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:54 2022] GET /identity/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:54.887282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:54.986883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8cee9-bfdb-4aad-b648-8415a7991c39 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:55.020611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:55.021272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:55.021552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:55.021828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:55.022544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=7cbb048dcc364049b9a2a1a4fa18ce6f, user.id=7e5d31d4baf34cddb93a2f0adfef85d2) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:55.022846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:55.024273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8778d4c-5fd0-401e-8dee-8bca74fc24e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:55.029623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 570/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:02:54 2022] GET /identity/v3/role_assignments?user.id=7e5d31d4baf34cddb93a2f0adfef85d2&scope.project.id=7cbb048dcc364049b9a2a1a4fa18ce6f&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:56.900983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef813-1c08-4673-9b4a-d93554652de3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:56.901559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef813-1c08-4673-9b4a-d93554652de3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:56.901559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ef813-1c08-4673-9b4a-d93554652de3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:56.902894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 577/1147] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:56.908061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:56.908446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:56.908602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:56.926293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:56.927975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:56.981382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:56.992008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 571/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:02:57.254802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ef813-1c08-4673-9b4a-d93554652de3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:57.354731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ef813-1c08-4673-9b4a-d93554652de3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:57.380490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:57.388114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:57.388723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:57.389001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:57.389308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:57.399011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:02:57.403730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:02:57.404834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:02:57.405955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 578/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:02:57.410802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:57.506228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c4667-68ab-4981-b252-a65fa67b72d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:57.539131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:57.539720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:57.539996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:57.540267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:57.540917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:02:57.541244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:57.542597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:57.550247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 572/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:57.556759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:57.652869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591b8f-edd0-4261-9dfc-1bc6824d6a33 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:57.685333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:57.685933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:57.686210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:57.686481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] PATH_INFO: `/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:57.687595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5d31d4baf34cddb93a2f0adfef85d2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:57.689011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:57.690575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 579/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:02:57 2022] GET /identity/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:57.696115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:57.749810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:57.794396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed8185-e0ef-49e1-9638-c73407e62725 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:57.826137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:57.826755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:57.827032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:57.827307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] PATH_INFO: `/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:57.828416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] RBAC: Authorizing `identity:get_project(project_id=7cbb048dcc364049b9a2a1a4fa18ce6f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:57.829884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da668c-ac9b-4b07-bea7-595cb35ff710 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:57.831508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 573/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:02:57 2022] GET /identity/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:02:57.837869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:02:57.897265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:02:57.939468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db645-b233-4dc2-a65f-d1150c338c24 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:02:57.971184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:02:57.971817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:57.972090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:57.972357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] PATH_INFO: `/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f/users/7e5d31d4baf34cddb93a2f0adfef85d2/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:57.974823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cbb048dcc364049b9a2a1a4fa18ce6f, user_id=7e5d31d4baf34cddb93a2f0adfef85d2, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:02:57.976231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b3f41-cbed-4b6c-b2a8-2c48166bdfe0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:02:57.989143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 580/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:02:57 2022] PUT /identity/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f/users/7e5d31d4baf34cddb93a2f0adfef85d2/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:02:59.854364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fd8c3-8b83-4fa6-bf5d-8b3805544396 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:59.854885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fd8c3-8b83-4fa6-bf5d-8b3805544396 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:59.854994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fd8c3-8b83-4fa6-bf5d-8b3805544396 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:59.856261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 574/1154] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:02:59.861249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:02:59.861683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:02:59.861800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:02:59.882775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:02:59.883990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:02:59.947633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 581/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:02:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:00.216390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fd8c3-8b83-4fa6-bf5d-8b3805544396 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:00.314196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fd8c3-8b83-4fa6-bf5d-8b3805544396 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:00.336072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:00.347266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:00.347875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:00.348151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:00.348418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:00.358541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:03:00.363237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:03:00.364323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:03:00.365537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 575/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:03:00.370480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:00.415111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:00.471051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d8540-bb4e-4f95-8639-5006a166e6d6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:00.502874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:00.503463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:00.503743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:00.504016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:00.504672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:00.504974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:00.506876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:00.514496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 582/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:00.520153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:00.616520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870552b6-4740-440b-a00b-0458edac6e53 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:00.648909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:00.649547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:00.649826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:00.650098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] PATH_INFO: `/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:00.651215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] RBAC: Authorizing `identity:get_user(user_id=7e5d31d4baf34cddb93a2f0adfef85d2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:00.652645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:00.654237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 576/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:03:00 2022] GET /identity/v3/users/7e5d31d4baf34cddb93a2f0adfef85d2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:00.660148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:00.757330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0424f-97b7-4029-b6bc-387e0cd44c25 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:00.766245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:00.789965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:00.790562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:00.790840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:00.791112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:00.856151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:00.857619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:00.951889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 583/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:03:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:03:00.957804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:01.006648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:01.055112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442b81f-b8a7-445a-b2b5-64e18716d5ca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:01.086486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:01.087091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:01.087374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:01.087648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] PATH_INFO: `/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:01.088753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] RBAC: Authorizing `identity:get_project(project_id=7cbb048dcc364049b9a2a1a4fa18ce6f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:01.090202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c53c1f-f011-4c03-9d10-f6568580b7d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:01.091812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 577/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:03:00 2022] GET /identity/v3/projects/7cbb048dcc364049b9a2a1a4fa18ce6f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:01.097891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:01.108115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:01.192960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4419d8e1-ac3c-49e7-9112-6e9807230590 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:01.224902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:01.225507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:01.225781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:01.226044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:01.226752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=7cbb048dcc364049b9a2a1a4fa18ce6f, user.id=7e5d31d4baf34cddb93a2f0adfef85d2) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:01.227051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:01.228445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891e6ac-5ccf-4330-9bb2-7e8d234f683b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:01.233928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 584/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:03:01 2022] GET /identity/v3/role_assignments?user.id=7e5d31d4baf34cddb93a2f0adfef85d2&scope.project.id=7cbb048dcc364049b9a2a1a4fa18ce6f&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:03:03.157437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762669-9b2f-4a4a-a258-ec9bcfe2a5f1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:03.157991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762669-9b2f-4a4a-a258-ec9bcfe2a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:03.157991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24762669-9b2f-4a4a-a258-ec9bcfe2a5f1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:03.159308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 578/1162] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:03.165008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:03.165470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:03.165588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:03.183762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:03.184944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:03.248717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 585/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:03.513300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24762669-9b2f-4a4a-a258-ec9bcfe2a5f1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:03.610030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24762669-9b2f-4a4a-a258-ec9bcfe2a5f1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:03.626569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:03.643149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:03.643740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:03.644014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:03.644283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:03.709217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:03.710644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:03.806802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 579/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:03:03.812690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:03.814866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:03.909902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7525f05-b91e-4416-b07b-605f19c67ff0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:03.943519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:03.944135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:03.944415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:03.944687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:03.951206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:03.952634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d933ab57-03cc-435a-9406-81a48e695737 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:03:03.963797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:03:03.964807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:03:03.965777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 586/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:03:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:03:03.970858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:03.989026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:04.067442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a45dd69-04bb-46cb-ab23-1d308ea1583c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:04.099865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:04.100451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:04.100727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:04.100998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:04.101655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:04.101955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:04.103316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64373bb-62f8-45f2-b6e0-8b64572bebf2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:04.110161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 580/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Sep 1 07:03:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:05.982818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c778b-0f7a-4fcf-bedb-9abd0cba2557 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:05.983328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c778b-0f7a-4fcf-bedb-9abd0cba2557 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:05.983328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c778b-0f7a-4fcf-bedb-9abd0cba2557 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:05.984673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 587/1167] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:05.990173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2f48-8562-46c9-93eb-eb5167a77e98 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:05.990586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2f48-8562-46c9-93eb-eb5167a77e98 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:05.990726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2f48-8562-46c9-93eb-eb5167a77e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:06.009187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-db0e2f48-8562-46c9-93eb-eb5167a77e98 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:06.010366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db0e2f48-8562-46c9-93eb-eb5167a77e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:06.073949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 581/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:06.342401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c778b-0f7a-4fcf-bedb-9abd0cba2557 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:06.403746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:06.442030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c778b-0f7a-4fcf-bedb-9abd0cba2557 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:06.473533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:06.474639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:06.474917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:06.475190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:06.475849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:06.477216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296afebe-f4b8-45ab-b9c1-d9d945495bb5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:06.492420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 588/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu Sep 1 07:03:06 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:03:08.424489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13e670-79f2-48dc-b92a-bb6804a42e6b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:08.425063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13e670-79f2-48dc-b92a-bb6804a42e6b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:08.425063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13e670-79f2-48dc-b92a-bb6804a42e6b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:08.426408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 582/1170] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:08.431434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:08.431849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:08.431982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:08.449638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:08.450823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:08.515738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 589/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:08.788336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13e670-79f2-48dc-b92a-bb6804a42e6b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:08.803454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:08.898994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13e670-79f2-48dc-b92a-bb6804a42e6b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:08.931871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:08.932485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:08.932759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:08.933028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] PATH_INFO: `/v3/domains/0f07e866ad754d379a82d146a0260231` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:08.942939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0f07e866ad754d379a82d146a0260231)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:08.944365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b6dd4-32f3-404c-8e14-fd9c2f184ef0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:08.945970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 583/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Sep 1 07:03:08 2022] GET /identity/v3/domains/0f07e866ad754d379a82d146a0260231 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:08.951814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:09.002254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:09.048159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ee961-8ffe-4582-ae29-da1a9083d106 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:09.079678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:09.080270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:09.080547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:09.080817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:09.081483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:09.082895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c71d1-1361-4350-a6f0-50985f7c54ad None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:09.109558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 590/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:03:08 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:03:11.007466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d4cc5-60b8-45c7-ab03-43abba48cbd2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:11.008015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d4cc5-60b8-45c7-ab03-43abba48cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:11.008015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d4cc5-60b8-45c7-ab03-43abba48cbd2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:11.009376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 584/1174] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:11.014371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:11.014764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:11.014965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:11.033986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:11.035153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:11.098775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 591/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:11.363093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d4cc5-60b8-45c7-ab03-43abba48cbd2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:11.433353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:11.463720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d4cc5-60b8-45c7-ab03-43abba48cbd2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:11.496585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:11.497221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:11.497504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:11.497779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] PATH_INFO: `/v3/domains/0f07e866ad754d379a82d146a0260231` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:11.498881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=0f07e866ad754d379a82d146a0260231)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:11.500311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb47d69-3a60-49ab-ae82-08508bf390ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:11.501890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 585/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Sep 1 07:03:11 2022] GET /identity/v3/domains/0f07e866ad754d379a82d146a0260231 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:11.507414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:11.539563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:11.604248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051636b7-3116-4ec4-8eb4-4580c9937ddd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:11.637613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:11.638152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:11.638432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:11.638706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:11.639359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:11.640769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f5cb6-a5de-4dc5-af1f-2b9c007fbaf3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:11.661698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 592/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:03:11 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:03:13.570814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395d07b-e939-4416-88a2-078073efe37d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:13.571370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395d07b-e939-4416-88a2-078073efe37d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:13.571370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395d07b-e939-4416-88a2-078073efe37d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:13.572699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 586/1178] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:13.577663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:13.578088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:13.578211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:13.596914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:13.598293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:13.663057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 593/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:13.930115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395d07b-e939-4416-88a2-078073efe37d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.034950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395d07b-e939-4416-88a2-078073efe37d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.066343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.066962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.067238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.068027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.075293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:14.078849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:03:14.084396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:03:14.085594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:03:14.086532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 587/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:03:14.091564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.111739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:14.188335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e27ea1-15f0-4986-99f8-018c923c0023 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.222062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.222661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.222939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.223215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.223869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:14.224171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:14.225613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:14.233471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 594/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:14.239895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.334842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c7dc7-2fc4-4e34-8845-0fa9b7d1d32b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.366475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.367080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.367353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.367621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] PATH_INFO: `/v3/users/55c8c538bdc849358ddd7290e35a1c30` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.381247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] RBAC: Authorizing `identity:get_user(user_id=55c8c538bdc849358ddd7290e35a1c30)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:14.382693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:14.384304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 588/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:03:14 2022] GET /identity/v3/users/55c8c538bdc849358ddd7290e35a1c30 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:14.390054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.469911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:14.488099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7075a-666b-459b-a26c-0f741cd08261 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.520645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.521265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.521545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.521817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.585227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:14.586666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:14.663769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:14.681823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 595/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:03:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:03:14.687627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.743794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:14.785151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf6959-6d68-421a-be09-d7f6b39a0d2a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.817934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.818603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.818883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.819158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] PATH_INFO: `/v3/projects/82915c16da7c40cfabb6f5aa54db22c1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.820324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] RBAC: Authorizing `identity:get_project(project_id=82915c16da7c40cfabb6f5aa54db22c1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:14.821799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579f3f-8c58-45d2-a51b-fe3ea14afa70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:14.823436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 589/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:03:14 2022] GET /identity/v3/projects/82915c16da7c40cfabb6f5aa54db22c1 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:14.829479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:14.932775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d866d44-6d34-46bd-8346-b3257101afba None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:14.968323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:14.968916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:14.969216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:14.969487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:14.970205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=82915c16da7c40cfabb6f5aa54db22c1, user.id=55c8c538bdc849358ddd7290e35a1c30) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:14.970509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:14.972523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70f7a1-f49c-4c35-9455-435747ba7f35 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:14.978686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 596/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:03:14 2022] GET /identity/v3/role_assignments?user.id=55c8c538bdc849358ddd7290e35a1c30&scope.project.id=82915c16da7c40cfabb6f5aa54db22c1&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:16.835185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c6d-ed58-4b4b-bc3d-58c6b8642ce1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:16.835674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c6d-ed58-4b4b-bc3d-58c6b8642ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:16.835780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c6d-ed58-4b4b-bc3d-58c6b8642ce1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:16.837067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 590/1186] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:16.842076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:16.842498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:16.842615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:16.860687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:16.861897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:16.905209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:16.929943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 597/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:17.197960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f379c6d-ed58-4b4b-bc3d-58c6b8642ce1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:17.297471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f379c6d-ed58-4b4b-bc3d-58c6b8642ce1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:17.322628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:17.331230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:17.331843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:17.332120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:17.332389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:17.342166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:03:17.346973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:03:17.348129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:03:17.349185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 591/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:03:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:03:17.354115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:17.447840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070d455-b423-4661-a6a1-36201e0b53bc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:17.479795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:17.480317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:17.480589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:17.480853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:17.481525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:17.481821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:17.483159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:17.491195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 598/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:17.497631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:17.592166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64f792-84f7-4af9-ba55-30d121de493b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:17.625533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:17.626142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:17.626419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:17.626692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] PATH_INFO: `/v3/users/55c8c538bdc849358ddd7290e35a1c30` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:17.627802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] RBAC: Authorizing `identity:get_user(user_id=55c8c538bdc849358ddd7290e35a1c30)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:17.629253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:17.630998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 592/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:03:17 2022] GET /identity/v3/users/55c8c538bdc849358ddd7290e35a1c30 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:17.636535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:17.674884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:17.732725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57daeb-bb53-4ea6-b3b1-49e106a982fb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:17.766024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:17.766147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:17.766341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:17.766607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] PATH_INFO: `/v3/projects/82915c16da7c40cfabb6f5aa54db22c1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:17.767700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] RBAC: Authorizing `identity:get_project(project_id=82915c16da7c40cfabb6f5aa54db22c1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:17.769108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451c8b3-4ede-4503-a98c-1bf14a9888da None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:17.770778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 599/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:03:17 2022] GET /identity/v3/projects/82915c16da7c40cfabb6f5aa54db22c1 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:17.776643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:17.830596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:17.873795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b9da1-bdf3-4ebc-ab3e-b3374fc3be4d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:17.906467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:17.907105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:17.907380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:17.907652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] PATH_INFO: `/v3/projects/82915c16da7c40cfabb6f5aa54db22c1/users/55c8c538bdc849358ddd7290e35a1c30/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:17.909546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] RBAC: Authorizing `identity:create_grant(project_id=82915c16da7c40cfabb6f5aa54db22c1, user_id=55c8c538bdc849358ddd7290e35a1c30, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:17.910967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59880602-6c09-49ed-8920-60e2cf27d168 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:17.923323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 593/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:03:17 2022] PUT /identity/v3/projects/82915c16da7c40cfabb6f5aa54db22c1/users/55c8c538bdc849358ddd7290e35a1c30/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:03:19.770156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2bc2-3a8a-4c43-ab05-560469f75508 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:19.770606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2bc2-3a8a-4c43-ab05-560469f75508 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:19.770737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a2bc2-3a8a-4c43-ab05-560469f75508 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:19.772063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 600/1193] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:03:19.777189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:19.777557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:19.777742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:19.795828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:03:19.797003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:03:19.860229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 594/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:03:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:03:20.128627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a2bc2-3a8a-4c43-ab05-560469f75508 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:20.226998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a2bc2-3a8a-4c43-ab05-560469f75508 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:20.234768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:20.261143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:20.261786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:20.262128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:20.262451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:20.272602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:03:20.277308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:03:20.278416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:03:20.279470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 601/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:03:20.284310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:20.327379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:20.379916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c39b0-c1e1-4472-b649-96ee0d1f136d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:20.412660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:20.413274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:20.413549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:20.413817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:20.414463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:20.414759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:20.416130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:20.423873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 595/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:20.429602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:20.525858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2db9-9abc-465e-85dd-865f1455e627 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:20.557092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:20.557717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:20.557991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:20.558257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] PATH_INFO: `/v3/users/55c8c538bdc849358ddd7290e35a1c30` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:20.559343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] RBAC: Authorizing `identity:get_user(user_id=55c8c538bdc849358ddd7290e35a1c30)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:20.560753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:20.562303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 602/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Sep 1 07:03:20 2022] GET /identity/v3/users/55c8c538bdc849358ddd7290e35a1c30 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:20.568490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:20.665065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c750ac-ad6f-4729-85d0-1e9afee3e6ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:20.671682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:20.698503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:20.699094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:20.699370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:20.699634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:20.763746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:20.765237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:20.859625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 596/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:03:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:03:20.866030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:20.900596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:20.965647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bcbe2-844a-4600-8745-9e8414b3c0cd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:20.997940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:20.998550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:20.998831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:20.999105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] PATH_INFO: `/v3/projects/82915c16da7c40cfabb6f5aa54db22c1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:21.000211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] RBAC: Authorizing `identity:get_project(project_id=82915c16da7c40cfabb6f5aa54db22c1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:21.001674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19f2b8-1fa2-480f-b26a-90584a5c8315 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:21.003274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 603/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Sep 1 07:03:20 2022] GET /identity/v3/projects/82915c16da7c40cfabb6f5aa54db22c1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:21.009970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:03:21.018018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:21.104836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3ebeb-0c5d-4b60-be5f-628962ae0d85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:03:21.138885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:03:21.139479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:03:21.139756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:03:21.140026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:03:21.140740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=82915c16da7c40cfabb6f5aa54db22c1, user.id=55c8c538bdc849358ddd7290e35a1c30) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:03:21.141044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:03:21.142501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6801de-6046-47d5-8e98-3dbc151c0dfe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:03:21.148051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 597/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:03:21 2022] GET /identity/v3/role_assignments?user.id=55c8c538bdc849358ddd7290e35a1c30&scope.project.id=82915c16da7c40cfabb6f5aa54db22c1&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:03:30.993580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=49 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:03:31.146123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=65 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:16.019382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ceefe-2f27-4d7e-8ce2-8de9295005f1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:16.020680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ceefe-2f27-4d7e-8ce2-8de9295005f1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:16.020680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ceefe-2f27-4d7e-8ce2-8de9295005f1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:16.021678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 604/1201] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:16.026862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:16.027346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:16.027467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:16.052567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:16.053856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:16.132465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 598/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 07:04:16.400769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ceefe-2f27-4d7e-8ce2-8de9295005f1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:16.532632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ceefe-2f27-4d7e-8ce2-8de9295005f1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:16.573193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:16.573863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:16.574138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:16.574432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:16.588278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:16.590354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:16.607240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:16.608359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:16.609473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:16.609473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:16.609473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:16.609473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 605/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:16.617732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:16.772229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31cd1-6f76-4911-a86c-ff656fb24cd7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:16.820956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:16.828886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:16.830050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:16.830367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:16.830650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:16.831504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:16.831946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:16.833871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be0e19-8e1a-4d08-b52a-5283a2595634 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:16.849789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 599/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:16.855641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:16.927262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:16.987175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da23305-cbdc-4d67-b54b-15be7c17495f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:17.033950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:17.035331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:17.035608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:17.035959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:17.036587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:17.038034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a5b75-6ace-477d-970c-a8bc17919d9f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:17.069180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 606/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Sep 1 07:04:16 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:04:19.493337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd9b9-1225-4a1e-808a-294c0673ef95 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:19.493889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd9b9-1225-4a1e-808a-294c0673ef95 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:19.493889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bd9b9-1225-4a1e-808a-294c0673ef95 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:19.495372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 600/1206] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:19.500968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:19.501478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:19.501590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:19.520746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:19.522059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:19.594186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 607/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:19.872123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bd9b9-1225-4a1e-808a-294c0673ef95 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:19.986743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bd9b9-1225-4a1e-808a-294c0673ef95 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.028773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.029593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.029909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.030269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.042054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:20.047055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:20.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:20.050407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 601/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:04:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:04:20.054744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:20.098923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:20.162225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9414e-f7a6-4bab-9dc3-51ee26f70267 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.197436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.198050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.198333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.198602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.199269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:20.199575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:20.200944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:20.209336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 608/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:20.214994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:20.275052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:20.331418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34661b32-cf9d-41b6-8a86-1947cd7d8651 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.370405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.371034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.371550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.371836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.384024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:20.385553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:20.400287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:20.401508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:20.402614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:20.402614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:20.402614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:20.404291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 602/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:20.409859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:20.513079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:20.516701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82043d74-feb6-4167-bd5f-ec77b1dfb6e9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.558483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.559100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.559406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.559635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.560278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:20.560580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:20.561999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:20.575577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 609/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:20.581322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:20.692238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcdacb-5b3e-4dce-aff4-c060519038da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.693930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:20.729865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.730508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.730827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.731100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.740209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:20.741837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:20.754252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:20.755535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:20.756724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:20.762138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 603/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:20.765304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:20.879781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650009b6-4a1b-4bdd-bf2e-6f0f854a5d77 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:20.916950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:20.917663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:20.917982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:20.918270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:20.919111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:20.919474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:20.921070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:20.945323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 610/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:20.946591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:21.080864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f9b164-7392-4036-963c-de071a2c6388 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:21.122124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:21.123111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:21.123585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:21.123882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:21.158930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:21.202017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:21.203700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:21.319413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 604/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:04:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:04:21.325070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:21.355174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:21.445883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7d13a-1f4b-40d8-b43f-b95627a8afad None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:21.486637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:21.487739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:21.488266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:21.488954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:21.502271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:21.504565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:21.521898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:21.523460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:21.524583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:21.524583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:21.524583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:21.530770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 611/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:21.531705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:21.599009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:21.643678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64315e9-91a4-4040-8b13-b3b95f99727c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:21.679383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:21.680016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:21.680313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:21.680568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:21.681240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:21.681541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:21.682917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:21.695567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 605/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:21.701482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:21.795994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:21.836884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe7b4e-a9ab-40e1-89f3-1c0cad42ad2d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:21.872336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:21.872991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:21.873347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:21.873676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:21.884066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:21.885271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:21.898180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:21.899302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:21.900245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 612/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:21.905586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:22.016597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf574f-4407-4e76-846e-289b3a61ca65 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:22.043413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:22.059780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:22.060709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:22.061090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:22.061563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:22.062480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:22.062842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:22.064628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5a20d-465c-47e5-9d17-c55cf5e705df None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:22.080872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 606/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:04:22.086885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:22.212916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4ce7-5667-42a0-a559-c3cadb9f9c60 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:22.243114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:22.249808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:22.250409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:22.250708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:22.251002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:22.251897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=25fbbb7831ed429cb38cd7918e1923ec) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:22.252417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:22.254903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fd927-62b8-4b29-9512-af569d9e414b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:22.261443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 613/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:04:22 2022] GET /identity/v3/role_assignments?user.id=25fbbb7831ed429cb38cd7918e1923ec&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:24.157140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd811-cfe6-41e6-9f0a-3b3a8acd07da None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:24.157716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd811-cfe6-41e6-9f0a-3b3a8acd07da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:24.157716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd811-cfe6-41e6-9f0a-3b3a8acd07da None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:24.159755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 607/1220] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:24.165049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:24.165544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:24.165639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:24.185071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:24.186373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:24.263564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 614/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:24.563204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd811-cfe6-41e6-9f0a-3b3a8acd07da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:24.681358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd811-cfe6-41e6-9f0a-3b3a8acd07da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:24.715976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:24.716615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:24.716891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:24.717253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:24.728678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:24.733535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:24.737187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:24.738430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 608/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:24.740649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:24.846134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1f2e2-e7fe-43c2-8d0e-3d64d27d8ed5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:24.871643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:24.883955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:24.884553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:24.884842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:24.885090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:24.885772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:24.886071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:24.887426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:24.895664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 615/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:24.901776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:24.946139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:25.002968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefd795-66e5-41be-ba97-5c51177e4423 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.036902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.037568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.037847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.038117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.048624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.050149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:25.064943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:25.066144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:25.067778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:25.067778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:25.067778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:25.067778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 609/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:25.072460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.178491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ee1bc-8324-4f95-a4cd-cb6d106437dd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.219534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.220143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.220426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.220702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.221372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:25.221677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.223041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.239327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 616/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:04:25.245331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.359675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:25.370706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ae67a-b8d6-4a89-9196-f7927bd2aac3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.410643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.411284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.411554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.411818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.419903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.422350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:25.436307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:25.437605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:25.444100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 610/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:25.444853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.502705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:25.560202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab501c0-c0bf-47b9-99da-4515db263e7b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.602765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.603375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.603671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.603943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.604666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:25.604993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.607099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.627254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 617/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:04:25.633169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.739034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fee0aa-2eaa-4205-8fd2-b9a0bc8aa7b3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.775234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.775872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.776155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.776471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.788488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.790100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:25.807100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:25.808294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:25.809454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:25.809454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:25.809454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:25.809454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 611/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:25.814451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.924331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d7cf8-c29e-4ef7-918f-848375a0624c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:25.926677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:25.962602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:25.963285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:25.963585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:25.963868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:25.964542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:25.964835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:25.966348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:25.978811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 618/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:25.984757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:25.995387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:26.089991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f250-396b-427e-a02f-c8107d25e789 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:26.127983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:26.128622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:26.129477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:26.129748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:26.136908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:26.138409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:26.150362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:26.151598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:26.152877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 612/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 07:04:26.157526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:26.267094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275c2a3-68ae-4d9f-a79d-a5c11f64ee10 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:26.306324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:26.306932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:26.307216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:26.307491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:26.308214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:26.308491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:26.309989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3e735-78b8-4db4-9a08-b3213a5b12af None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:26.322763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 619/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:26.329250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:26.399338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:26.436386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77355457-1427-427d-9b01-ef560f4398d4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:26.473696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:26.474442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:26.474739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:26.475009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/25fbbb7831ed429cb38cd7918e1923ec/roles/150d00c50e934e5fa1b1d8c5f4a0754d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:26.490548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=25fbbb7831ed429cb38cd7918e1923ec, role_id=150d00c50e934e5fa1b1d8c5f4a0754d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:26.492095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b7f19-a316-4a13-89db-5b2715638e76 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:26.505297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 613/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:04:26 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/25fbbb7831ed429cb38cd7918e1923ec/roles/150d00c50e934e5fa1b1d8c5f4a0754d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:04:28.469816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2350f-bf4f-437b-bb21-8f8f4237a37d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:28.470387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2350f-bf4f-437b-bb21-8f8f4237a37d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:28.470387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2350f-bf4f-437b-bb21-8f8f4237a37d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:28.471685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 620/1233] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:28.477552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:28.477983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:28.478176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:28.498804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:28.499992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:28.570259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 614/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:28.855529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2350f-bf4f-437b-bb21-8f8f4237a37d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:28.882319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:28.961773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2350f-bf4f-437b-bb21-8f8f4237a37d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:28.998613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:28.999239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:28.999517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:28.999787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.010349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:29.015582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:29.016685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:29.017814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 621/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Sep 1 07:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:29.023941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:29.074069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:29.131545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847f079-3d1b-4c98-b24c-f72575555165 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:29.166785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224af82-a969-4e95-a031-08726408805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:29.167400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224af82-a969-4e95-a031-08726408805f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:29.167677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224af82-a969-4e95-a031-08726408805f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:29.167948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224af82-a969-4e95-a031-08726408805f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.168608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224af82-a969-4e95-a031-08726408805f None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:29.168911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224af82-a969-4e95-a031-08726408805f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:29.170364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224af82-a969-4e95-a031-08726408805f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:29.178440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 615/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:29.184291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:29.278915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:29.304402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7f9b0-b4c1-4d75-9429-aa6c8efc3607 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:29.342642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:29.343267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:29.343542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:29.343810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.353945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:29.355387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:29.370034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:29.371062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:29.372721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:29.372721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:29.372721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:29.372721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 622/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:29.377345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224af82-a969-4e95-a031-08726408805f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:29.489965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224af82-a969-4e95-a031-08726408805f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:29.502473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:29.529239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:29.529782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:29.530059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:29.530333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.531459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:29.531763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:29.533226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:29.550444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 616/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:29.552427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:29.660663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a09da9-162b-4a6f-9425-bfeb564ada29 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:29.697913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:29.698523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:29.698799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:29.699073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.701978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:29.707321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:29.709268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:29.720553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:29.721756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:29.722732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 623/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:29.727895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:29.836488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff0636-65d5-4fb7-95b0-094890ef625a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:29.874018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:29.874797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:29.875109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:29.875394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:29.876345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:29.876682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:29.878362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:29.898314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 617/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:29.904367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:30.013714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06ea7a-5e4e-49f2-bdc6-09d8439496e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:30.051997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:30.052632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:30.052901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:30.053268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:30.121160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:30.128453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:30.130206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:30.240485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 624/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:04:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:04:30.247311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:30.281216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:30.362857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76506354-b82f-48bc-a124-c991f525bfa4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:30.397046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:30.397638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:30.397911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:30.398176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:30.408492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:30.410096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:30.425577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:30.427118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:30.428586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:30.428586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:30.428586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:30.428586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 618/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:30.432644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:30.534440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:30.541320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f66bd4-42f5-4bce-a183-97fb1b04d02f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:30.577195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:30.577843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:30.578067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:30.578341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:30.578974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:30.579288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:30.580907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:30.594749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 625/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:30.601158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:30.701753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:30.724006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ce6c-d92e-403b-a5fa-c31999358d46 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:30.764490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:30.765306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:30.766019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:30.766130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:30.775366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:30.777156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:30.791109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:30.792219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:30.793360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 619/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:30.798102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:30.915740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2f1d0-93b9-46e2-ac1e-b9e8276e457f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:30.954186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:30.954715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:30.954991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:30.955262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:30.955934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:30.956242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:30.958389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270a2c7-7f64-43a8-87ab-7b80bfe64321 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:30.979087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 626/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:30.980473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:31.088105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c66c60-57d5-48ff-b9b2-9143ef7c9716 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:31.124336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:31.125029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:31.125343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:31.125655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:31.126464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] RBAC: Adding query filter params (role.id=150d00c50e934e5fa1b1d8c5f4a0754d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=25fbbb7831ed429cb38cd7918e1923ec) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:31.126770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:31.128344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f45775-6224-41b2-8782-7a3e36948af8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:31.134963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 620/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:04:30 2022] GET /identity/v3/role_assignments?user.id=25fbbb7831ed429cb38cd7918e1923ec&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=150d00c50e934e5fa1b1d8c5f4a0754d => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:33.258862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e64c6-0cde-4a62-9243-e9f2bb197abc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:33.259424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e64c6-0cde-4a62-9243-e9f2bb197abc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:33.259543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e64c6-0cde-4a62-9243-e9f2bb197abc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:33.261138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 627/1247] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:33.266931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:33.267371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:33.267555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:33.275195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:33.289225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:33.290440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:33.377031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 621/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:33.720418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e64c6-0cde-4a62-9243-e9f2bb197abc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:33.741816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:33.835060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e64c6-0cde-4a62-9243-e9f2bb197abc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:33.872745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:33.873396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:33.873688 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:33.873966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:33.884700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:33.890904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:33.892038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:33.893281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 628/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:33.898847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.014200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab5a27-86f3-4239-8e70-95766620041e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.050679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:34.056948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.057612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.057897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.058168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:34.058853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:34.059488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:34.060982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:34.070650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 622/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:34.077423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.164425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:34.192595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e4dba-401a-4c47-acde-9e22cfe2b159 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.234309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.235001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.235334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.235503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:34.251993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:34.255328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:34.279373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:34.281043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:34.282244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:34.282244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:34.282244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:34.287922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 629/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:34.289913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.412117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef1c52-a1e7-44f6-9987-743d5e0c5e9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.465570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.466573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.467034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.467457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:34.468494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:34.468948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:34.471079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:34.491737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 623/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:34.493512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.595619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5704c-1002-4bf7-9785-d4ed8fafdb72 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.623234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:34.632595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.633266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.633548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.633823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:34.641596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:34.643161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:34.657066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:34.658230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:34.659193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 630/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:34.664458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.729674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:34.782314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6758-086a-4c07-90cd-1042f85a7f32 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.819722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.820479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.820711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.820987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:34.821774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:34.822115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:34.824282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:34.844023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 624/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:34.850059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:34.960859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61994216-7706-424b-8335-d21e4b656b09 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:34.995142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:34.995899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:34.996302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:34.996928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:35.056457 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:35.073373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:35.075298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:35.186578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 631/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:04:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 07:04:35.192306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:35.307074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c9984-8e5f-4f3f-a4f4-799a67afab41 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:35.321566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:35.345471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:35.346184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:35.346463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:35.346740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:35.359011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:35.360802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:35.379561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:35.380745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:35.381828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:35.381828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:35.381828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:35.381828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 625/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:35.387619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:35.479095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:35.492397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b55f-4ccb-4ce6-96de-dbb804a89daa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:35.531249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:35.531855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:35.532131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:35.532402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:35.533048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:35.533398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:35.534813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:35.546690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 632/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:35.552639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:35.654240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e698aa-2810-42ab-90e0-025986379fb2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:35.692638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975183-9761-4158-85fe-82023133b77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:35.693324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975183-9761-4158-85fe-82023133b77e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:35.693659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975183-9761-4158-85fe-82023133b77e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:35.693881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975183-9761-4158-85fe-82023133b77e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:35.700634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975183-9761-4158-85fe-82023133b77e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:35.702110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975183-9761-4158-85fe-82023133b77e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c9975183-9761-4158-85fe-82023133b77e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:35.714061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:35.716102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:35.717073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 626/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:35.721375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:35.833229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9746f6e-bed4-45d7-bfb5-3d9726517efd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:35.867461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:35.868066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:35.868352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:35.868623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:35.869336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:35.869640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:35.871053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6911d9f-f457-4671-b685-6fa42e487890 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:35.884689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 633/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:35.891088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975183-9761-4158-85fe-82023133b77e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:35.920699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:36.005297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975183-9761-4158-85fe-82023133b77e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:36.041540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:36.042159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:36.042436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:36.042707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:36.043446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=25fbbb7831ed429cb38cd7918e1923ec) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:36.043748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:36.045224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef80ca-cd24-4de8-abd6-fd9cb45374b7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:36.065165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 627/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:04:35 2022] GET /identity/v3/role_assignments?user.id=25fbbb7831ed429cb38cd7918e1923ec&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:37.930699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696f6df-5dfd-440d-a1c1-e92720e088ac None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:37.931213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696f6df-5dfd-440d-a1c1-e92720e088ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:37.931354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696f6df-5dfd-440d-a1c1-e92720e088ac None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:37.932753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 634/1261] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:37.937950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:37.938368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:37.938503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:37.956328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:37.958108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:38.024322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 628/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:38.289553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696f6df-5dfd-440d-a1c1-e92720e088ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:38.300620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:38.388486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696f6df-5dfd-440d-a1c1-e92720e088ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:38.420721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:38.421378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:38.421658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:38.421924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:38.431581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:38.436227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:38.437365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:38.438255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 635/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:38.443777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:38.461944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:38.539914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f16c69-16bf-4094-ac14-02121b678815 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:38.572871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:38.573487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:38.573767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:38.574037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:38.574684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:38.574984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:38.576345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:38.584044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 629/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:04:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:38.589614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:38.653617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:38.686851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33443d44-cc95-426e-8841-e73e849bf5b9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:38.722365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:38.722997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:38.723284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:38.723538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:38.732966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:38.734408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:38.748940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:38.750044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:38.750984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:38.750984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:38.750984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:38.750984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 636/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:38.756434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:38.834046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:38.852846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e4d11-8ef2-4c30-b28b-48b55466a358 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:38.884917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:38.885534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:38.885810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:38.886081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:38.886735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:38.887032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:38.888389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:38.900102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 630/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:38.905672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.004698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355854bd-2e16-4803-94e6-6ec95019bb9c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.023402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:39.037418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.038027 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.038301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.038569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.045547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.047088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:39.058459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:39.059560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:39.060730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 637/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:39.065509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.162104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0cc15-066c-4ab9-98e4-f1baa7f71f7f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.194515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.195098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.195370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.195642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.196353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:39.196651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.198116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.206380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:39.214589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 631/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:39.220356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.328746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db1ac5-ef8f-498f-abf4-e8e0e405a838 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.366464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.367078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.367354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.367623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.377900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.379349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:39.392760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:39.393795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:39.395288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:39.395288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:39.395288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:39.395288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 638/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:39.400382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.496252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa40270-c2ac-46c9-8e3b-bdb9cbb7e2be None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.528979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.529599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.529874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.530140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.530765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:39.531061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.532416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.543693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 632/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:39.549408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.579612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:39.646731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580928a8-fb98-4972-b112-c3f017384b46 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.680036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.680655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.680934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.681236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.687665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.689107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:39.701064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:39.702159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:39.703127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 639/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:39.708510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.767658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:39.809006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18317-7ad6-41ac-b708-7ce509383f38 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.840630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.841252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.841529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.841797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.842481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:39.842784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.844210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49be1-3d78-4d04-a99f-95c5999b0d40 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:39.856120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 633/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:39.862095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:39.950600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:39.959776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac8951-a56e-4db4-ae02-f3bb394fd50f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:39.992827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:39.993499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:39.993780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:39.994053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] PATH_INFO: `/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/25fbbb7831ed429cb38cd7918e1923ec/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:39.995921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] RBAC: Authorizing `identity:create_grant(project_id=0bce5454d9704d3b989a162e84789f3b, user_id=25fbbb7831ed429cb38cd7918e1923ec, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:39.997644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686af25-f7db-4307-90b5-f3e8d91889ba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:40.009679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 640/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Sep 1 07:04:39 2022] PUT /identity/v3/projects/0bce5454d9704d3b989a162e84789f3b/users/25fbbb7831ed429cb38cd7918e1923ec/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:04:41.881088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217292-386b-442e-b985-f7dfbd3014a0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:41.881626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217292-386b-442e-b985-f7dfbd3014a0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:41.881740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4217292-386b-442e-b985-f7dfbd3014a0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:41.883014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 634/1274] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:41.888109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:41.888527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:41.888647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:41.907006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:41.908172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:41.971148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 641/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:42.245361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217292-386b-442e-b985-f7dfbd3014a0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:42.347363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4217292-386b-442e-b985-f7dfbd3014a0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:42.364195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:42.380453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:42.381064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:42.381366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:42.381637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:42.391241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 07:04:42.396514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:42.397629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:42.398674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 635/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Sep 1 07:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:42.405750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:42.482590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:42.500356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f167ce-4738-4e43-b6d7-4553032072b2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:42.532190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:42.532771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:42.533043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:42.533349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:42.533995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:42.534291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:42.536191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:42.543744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 642/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Sep 1 07:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:42.549358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:42.646958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cce57b-18bf-47dc-814b-e15ba640e8c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:42.678708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:42.679311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:42.679588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:42.679859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:42.689412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:42.690845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:42.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:42.706474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:42.708170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:42.708170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:42.708170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:42.708170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 636/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:42.712651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:42.811379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dfbf6-6117-489a-879d-484dc7eec8ea None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:42.843710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:42.844296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:42.844569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:42.844834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:42.845483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:42.845780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:42.847133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:42.858169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 643/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:42.863859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:42.894163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:42.962231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978aaa-9330-4837-affc-f2478968cb3f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:42.994367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:42.994982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:42.995261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:42.995533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.002977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.004991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 07:04:43.015974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:43.017096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:43.018108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 637/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:43.022891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.026587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:43.118253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e160d5e-4436-4e6a-a3a8-9c9101ba9eea None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:43.150879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:43.151469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:43.151747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:43.152016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.152732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:43.153036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.154473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.171781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 644/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:43.178050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.269690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:43.279316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805298e-ea05-44bc-a8b0-858278fa4a01 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:43.311747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:43.312349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:43.312629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:43.312904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.378073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.379545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.464438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:43.476799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 638/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 07:04:43.483183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.542656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:43.581191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fda43b-58d4-4284-9c01-35ac1f167fc0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:43.613500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:43.614111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:43.614388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:43.614660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.624454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.625931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 07:04:43.640334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:43.641477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 07:04:43.642549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 07:04:43.642549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 07:04:43.642549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:43.642549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 645/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:43.647697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.741619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c11e5-dc23-4d24-9e61-442a5f822796 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:43.775429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:43.776022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:43.776294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:43.776558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.777216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:43.777513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.778874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.789492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 639/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:43.795080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.890234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f831ef-f8a6-4ca2-8b7c-95d5e1b179bc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:43.903013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:43.921963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:43.922564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:43.922838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:43.923106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:43.929948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:43.931359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 07:04:43.942674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:43.943705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: ERROR keystone.server.flask.application Sep 01 07:04:43.944864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 646/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:43.949453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:43.997473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:44.045359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2ca8-5098-469f-824c-b7ffedc4b8d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:44.077745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:44.078331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:44.078606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:44.078875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:44.079548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:44.079849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:44.081292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad792-28df-4bc0-b10b-416b16d26dd6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:44.092551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 640/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Sep 1 07:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:44.099133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:44.194768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4fd8a-683a-4aa1-8312-e4250d08ec6f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:44.226652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:44.227232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:44.227506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:44.227772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:44.228478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] RBAC: Adding query filter params (role.id=9a0636d853ba4c5d9cdae17e523d036d, scope.project.id=0bce5454d9704d3b989a162e84789f3b, user.id=25fbbb7831ed429cb38cd7918e1923ec) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:44.228779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:44.230207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bac4e0-3153-41fe-b25c-dd64f94206f4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:44.235690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 647/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Sep 1 07:04:44 2022] GET /identity/v3/role_assignments?user.id=25fbbb7831ed429cb38cd7918e1923ec&scope.project.id=0bce5454d9704d3b989a162e84789f3b&role.id=9a0636d853ba4c5d9cdae17e523d036d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:46.127915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f220-6e92-4720-8856-7d2d6821c9f6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:46.128459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f220-6e92-4720-8856-7d2d6821c9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:46.128459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2f220-6e92-4720-8856-7d2d6821c9f6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:46.129843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 641/1288] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:46.134931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:46.135329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:46.135477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:46.153679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:46.154850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:46.183194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:46.219537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 648/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:46.492017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2f220-6e92-4720-8856-7d2d6821c9f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:46.590098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2f220-6e92-4720-8856-7d2d6821c9f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:46.600224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:46.624694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:46.625707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:46.625707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:46.626093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:46.626456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:46.628179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:46.638247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:46.639486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:46.640391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 642/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:04:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:46.645884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:46.739009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ed36-522b-45d1-bc5b-5492f4eaaf7a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:46.775699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:46.776295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:46.776586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:46.776833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:46.777478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:46.777774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:46.779131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffb46c-6258-4e07-b358-31b7b1451648 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:46.783713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 649/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 07:04:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:46.789418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:46.886013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df463cf-14f8-4331-9139-d05f2fd5bff1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:46.917652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:46.918240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:46.918519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:46.918791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:46.919409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:46.919709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:46.921067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df45bd-a7d0-49ce-ae0b-b6b8ed5a0311 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:46.925386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 643/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 07:04:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:48.792946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbe7e-6073-4d38-aef7-627c588af9e4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:48.793523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbe7e-6073-4d38-aef7-627c588af9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:48.793523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbe7e-6073-4d38-aef7-627c588af9e4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:48.794845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 650/1293] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:48.799915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0a09d-4cee-41bf-80a1-16830d5043d0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:48.800303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0a09d-4cee-41bf-80a1-16830d5043d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:48.800471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0a09d-4cee-41bf-80a1-16830d5043d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:48.818779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-02d0a09d-4cee-41bf-80a1-16830d5043d0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:48.819960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02d0a09d-4cee-41bf-80a1-16830d5043d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:48.848974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:48.885427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 644/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:49.153947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dbe7e-6073-4d38-aef7-627c588af9e4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:49.184334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:49.257130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dbe7e-6073-4d38-aef7-627c588af9e4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:49.295930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:49.296622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:49.296944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:49.297261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:49.298078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:49.299702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f7fac-9630-4558-8477-ab7b762fb86c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:49.316219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 651/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Sep 1 07:04:49 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:04:51.196647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888410b-2d20-437c-831f-4ee6f72dc5df None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:51.197214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888410b-2d20-437c-831f-4ee6f72dc5df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:51.197214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888410b-2d20-437c-831f-4ee6f72dc5df None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:51.198564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 645/1296] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:51.203656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:51.204077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:51.204205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:51.222362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:51.223537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:51.294900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 652/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:51.566340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888410b-2d20-437c-831f-4ee6f72dc5df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:51.665517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888410b-2d20-437c-831f-4ee6f72dc5df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:51.698168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:51.698780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:51.699059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:51.699332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:51.699906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:51.701297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:51.709932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:51.711088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:51.712127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 646/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:04:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:51.716829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:51.735011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:51.813384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d60d79-3405-480a-9d90-c52494ec0161 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:51.845056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:51.845611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:51.845884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:51.846151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:51.846789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:51.847086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:51.848981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:51.853856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 653/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 07:04:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:51.859886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:51.891474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:51.957839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379437b6-96ec-4aec-a93d-a39e798d9a2e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:51.990667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:51.991258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:51.991535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:51.991804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:51.992494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] RBAC: Adding query filter params (interface=public, service_id=24ba05980efe4cfab1c617af77f91e76, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:51.992799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:51.994189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb631d9-9f8e-42a7-90ec-7b1de192a10a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:51.999311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 647/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Sep 1 07:04:51 2022] GET /identity/v3/endpoints?service_id=24ba05980efe4cfab1c617af77f91e76&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:52.004910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:52.083868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:52.102037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc1bec-4039-49c1-a058-5fbf3f5a53e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:52.133701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:52.134292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:52.134570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:52.134838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:52.135428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:52.136781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436b04-96f6-4548-a7e0-8c224a39c437 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:52.143336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 654/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Sep 1 07:04:52 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:04:54.009831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff461e-a869-4d7d-a17b-33e37314a111 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:54.010406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff461e-a869-4d7d-a17b-33e37314a111 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:54.010406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff461e-a869-4d7d-a17b-33e37314a111 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:54.011729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 648/1302] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:04:54.016717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:54.017100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:54.017249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:54.034916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:04:54.036639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:04:54.100009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 655/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:04:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 07:04:54.380388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff461e-a869-4d7d-a17b-33e37314a111 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:54.471747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:54.479306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff461e-a869-4d7d-a17b-33e37314a111 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:54.510803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:54.511408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:54.511682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:54.511952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:54.512516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:54.513975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 07:04:54.522925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 07:04:54.523939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 01 07:04:54.525164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 649/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Sep 1 07:04:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 07:04:54.529853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:54.603210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:04:54.625479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b41b4d-4627-4f7a-b47d-45d95cf3ee59 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:54.659447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:54.659983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:54.660289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:54.660565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:54.661209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:04:54.661512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:54.662865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17cc0-a203-4813-98e1-7be269e5c5fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:54.667774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 656/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Sep 1 07:04:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:04:54.673512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:04:54.772556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b4898-81ce-4215-b56d-5e5329fa46d0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:04:54.806403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:04:54.807374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:04:54.807654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:04:54.807931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:04:54.808964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:04:54.810795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf264ac-adba-4521-88fd-e787d17f58d6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:04:54.830492 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 650/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:04:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:05:04.665030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=30 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:05:04.826095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:05:57.921709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea3b4-58c9-40f2-ab61-354c3c066140 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:05:57.922718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea3b4-58c9-40f2-ab61-354c3c066140 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:05:57.923141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea3b4-58c9-40f2-ab61-354c3c066140 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:05:57.924618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 657/1307] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:05:57.930398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751c4a3-192f-408a-8104-34d428801de0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:05:57.931235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751c4a3-192f-408a-8104-34d428801de0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:05:57.931656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751c4a3-192f-408a-8104-34d428801de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:05:58.077896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e751c4a3-192f-408a-8104-34d428801de0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:05:58.079199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e751c4a3-192f-408a-8104-34d428801de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:05:58.214110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 651/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:05:58.499387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:05:58.499662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:05:58.499938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:05:58.501697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 658/1309] 10.222.0.46 () {54 vars in 965 bytes} [Thu Sep 1 07:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:05:58.508746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd23c6-f6ca-4dfc-8087-1b0e931e036f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:05:58.509033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd23c6-f6ca-4dfc-8087-1b0e931e036f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:05:58.509347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd23c6-f6ca-4dfc-8087-1b0e931e036f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:05:58.610760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8bfd23c6-f6ca-4dfc-8087-1b0e931e036f None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:05:58.612452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8bfd23c6-f6ca-4dfc-8087-1b0e931e036f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:05:58.642242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:05:58.711277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 652/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Sep 1 07:05:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:05:58.720764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:05:58.723456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:05:58.857679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502ddf9-0189-40da-b9af-8633f8be61fb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:05:58.933221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:05:58.934490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:05:58.934946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:05:58.935363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:05:58.937129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:05:58.993367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:05:59.051059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:05:59.052783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db230aa-d200-4e1e-9430-7de5563591c4 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:05:59.073784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 659/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:01.310224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:01.310758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:01.310758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:01.312274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 653/1312] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:01.317799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790546-c05a-4521-8491-8c60fd234fd1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:01.318211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790546-c05a-4521-8491-8c60fd234fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:01.318359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790546-c05a-4521-8491-8c60fd234fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:01.405949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-69790546-c05a-4521-8491-8c60fd234fd1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:01.407147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69790546-c05a-4521-8491-8c60fd234fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:01.502162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 660/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:01.783491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:01.785166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:01.887438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73480c66-696a-4f8d-bea0-642a65e71d2d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:01.964592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:01.967723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:01.968342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:01.968838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:01.969176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:01.971003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:02.087806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:02.089283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42318028-296f-4573-a261-240138e6c3a1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:02.107609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 654/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:04.288971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:04.289580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:04.289580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:04.290871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 661/1315] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:04.295841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c027e9-f90e-4699-b0b3-813c37090b0f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:04.296231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c027e9-f90e-4699-b0b3-813c37090b0f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:04.296391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c027e9-f90e-4699-b0b3-813c37090b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:04.363156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e8c027e9-f90e-4699-b0b3-813c37090b0f None None] MFA Rules not processed for user `7c987b3959274c0893cd501e70f6d998`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:04.364342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e8c027e9-f90e-4699-b0b3-813c37090b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:04.413801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:04.457806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 655/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:04.739631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:04.741240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:04.751314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:04.844058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0bf2-9613-4fbb-83eb-df34fc980c24 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:04.916449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:04.917045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:04.917352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:04.917623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:04.919206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:04.983380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:05.029206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:05.030644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33584613-e264-414c-a626-f76d411668b9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:05.048958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 662/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:07.522145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:07.522806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:07.522806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:07.524059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 656/1318] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:07.529311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fec04-4818-44d8-a5a1-17bd6bb058f2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:07.529734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fec04-4818-44d8-a5a1-17bd6bb058f2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:07.529874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fec04-4818-44d8-a5a1-17bd6bb058f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:07.594114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-502fec04-4818-44d8-a5a1-17bd6bb058f2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:07.595323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-502fec04-4818-44d8-a5a1-17bd6bb058f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:07.690215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 663/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:07.975879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:07.977599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:08.078893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ceb50-9939-4558-a4c1-d4fb586d85c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:08.148705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:08.149325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:08.149600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:08.149866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:08.151456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:08.158646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:08.259256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:08.260616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590052e-f4b7-47ec-9630-3f09358d921e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:08.278281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 657/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:10.808550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:10.809082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:10.809082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:10.810534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 664/1321] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:10.815748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5658e-0763-493e-a675-7ddf23b650ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:10.816190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5658e-0763-493e-a675-7ddf23b650ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:10.816288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5658e-0763-493e-a675-7ddf23b650ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:10.882147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f8a5658e-0763-493e-a675-7ddf23b650ca None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:10.883343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8a5658e-0763-493e-a675-7ddf23b650ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:10.939802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:10.981314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 658/1322] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:11.259930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:11.261505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:11.280970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:11.364769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d301b-262a-40e0-88c5-04743d0ea5b3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:11.437760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:11.438282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:11.438597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:11.438832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:11.440960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:11.510844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:11.551639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:11.553068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc546f1-e2b0-4622-aec0-662d1671b1e3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:11.571390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 665/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:13.534647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e846da-af8d-4eae-97fb-6a1ba137803d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:13.535171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e846da-af8d-4eae-97fb-6a1ba137803d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:13.535171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e846da-af8d-4eae-97fb-6a1ba137803d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:13.536529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 659/1324] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:13.541693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69583c79-3aed-4a04-ae1d-1edd704bacbf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:13.542122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69583c79-3aed-4a04-ae1d-1edd704bacbf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:13.542238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69583c79-3aed-4a04-ae1d-1edd704bacbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:13.607323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-69583c79-3aed-4a04-ae1d-1edd704bacbf None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:13.608518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69583c79-3aed-4a04-ae1d-1edd704bacbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:13.712542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 666/1325] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:06:14.022455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:14.022832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:14.023175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:14.024604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 660/1326] 10.222.0.46 () {54 vars in 965 bytes} [Thu Sep 1 07:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:14.030797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-b514-408f-81d0-c5437eb4bf2a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:14.031147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-b514-408f-81d0-c5437eb4bf2a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:14.031489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-b514-408f-81d0-c5437eb4bf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:14.072233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:14.129429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ebed3ca8-b514-408f-81d0-c5437eb4bf2a None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:14.130720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ebed3ca8-b514-408f-81d0-c5437eb4bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:14.204577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 667/1327] 10.222.0.46 () {58 vars in 1062 bytes} [Thu Sep 1 07:06:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:06:14.213510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:14.215044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:14.320696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78d6b0-2916-424f-9e83-f40977f811fe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:14.393886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:14.394389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:14.394670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:14.394944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:14.396544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:14.413649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:14.506546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:14.508028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04abba3-d834-440e-8971-0d7532a1ad06 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:14.525691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 661/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:06:16.721228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:16.721768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:16.721768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:16.724392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 668/1329] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:16.729197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd771f3-b575-4354-b337-7c4005e4cde4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:16.729834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd771f3-b575-4354-b337-7c4005e4cde4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:16.729834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd771f3-b575-4354-b337-7c4005e4cde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:16.802577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bdd771f3-b575-4354-b337-7c4005e4cde4 None None] MFA Rules not processed for user `7c987b3959274c0893cd501e70f6d998`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:16.803816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bdd771f3-b575-4354-b337-7c4005e4cde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:16.867923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:16.900973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 662/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:17.190582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:17.192115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:17.299290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2072a3-74c3-49ae-a5da-5be61bed8b94 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:17.321732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:17.379067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:17.379678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:17.379955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:17.380224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:17.381864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:17.497307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:17.499009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca666d6-06cd-42f9-bc6f-6debedc6d290 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:17.517435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 669/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:19.849428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:19.849982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:19.849982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:19.851332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 663/1332] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:19.856384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d46c5-affc-413a-902c-2d76f01a8482 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:19.856822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d46c5-affc-413a-902c-2d76f01a8482 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:19.856930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d46c5-affc-413a-902c-2d76f01a8482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:19.920399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:19.923452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-922d46c5-affc-413a-902c-2d76f01a8482 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:19.925096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-922d46c5-affc-413a-902c-2d76f01a8482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:20.019441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 670/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:20.298940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:20.300487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:20.404048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686ead1-c5db-4616-abbe-24d609f18cbe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:20.476362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:20.476967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:20.477285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:20.477561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:20.479155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:20.506128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:20.588373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:20.589761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c383df-fafb-415e-9c33-4b32482d7751 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:20.607292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 664/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:22.539047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:22.539594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:22.539594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:22.540936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 671/1335] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:06:22.547072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332bdb-0bfa-4d92-920d-6dc0e59df567 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:22.547574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332bdb-0bfa-4d92-920d-6dc0e59df567 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:22.548076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38332bdb-0bfa-4d92-920d-6dc0e59df567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:22.615327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-38332bdb-0bfa-4d92-920d-6dc0e59df567 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:22.616518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38332bdb-0bfa-4d92-920d-6dc0e59df567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:22.688049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:22.712238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 665/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:22.994490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:22.996023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:23.096004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55987-5bca-4ccd-8e3c-4814d59d7b8a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:23.123646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:23.171287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:23.171893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:23.172169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:23.172442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:23.174052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:23.280411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:23.281861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cf47f-6bf6-4bd8-bca9-be7dcd461c15 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:23.299876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 672/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:25.648853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:25.649436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:25.649436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:25.650752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 666/1338] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:25.655836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbabe4-619f-4748-bb13-0c01f84f03d6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:25.656213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbabe4-619f-4748-bb13-0c01f84f03d6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:25.656397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbabe4-619f-4748-bb13-0c01f84f03d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:25.720907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bebbabe4-619f-4748-bb13-0c01f84f03d6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:25.722130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bebbabe4-619f-4748-bb13-0c01f84f03d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:25.730132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:25.820002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 673/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:26.097796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:26.099335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:26.202016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0ab0b-0eed-4e61-9b16-88def1d2e21d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:26.274268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:26.274876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:26.275153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:26.275424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:26.277009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:26.312073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:26.384163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:26.385636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308112de-77d1-4a82-87ca-cbc50af731de service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:26.403437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 667/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:30.306657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:30.307163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:30.307426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:30.308843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 674/1341] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:30.314178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaad8d2-5d7d-4613-b375-735cc1c13d10 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:30.314574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaad8d2-5d7d-4613-b375-735cc1c13d10 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:30.314741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaad8d2-5d7d-4613-b375-735cc1c13d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:30.381603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0eaad8d2-5d7d-4613-b375-735cc1c13d10 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:30.382821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0eaad8d2-5d7d-4613-b375-735cc1c13d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:30.468128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:30.479849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 668/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:30.765021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:30.767091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:30.871462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4afee6-4c8c-47b2-b994-8e4357be2051 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:30.908206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:30.944173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:30.944795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:30.945053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:30.945352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:30.946917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:31.053603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:31.055032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14aed4d-d73b-483b-9dfb-f10b0bea2aea service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:31.072919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 675/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:33.607869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:33.608433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:33.608433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:33.610023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 669/1344] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:33.615583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed634202-5f2c-4ae5-8a68-23a070c57f8b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:33.616003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed634202-5f2c-4ae5-8a68-23a070c57f8b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:33.616258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed634202-5f2c-4ae5-8a68-23a070c57f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:33.686836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ed634202-5f2c-4ae5-8a68-23a070c57f8b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:33.688053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed634202-5f2c-4ae5-8a68-23a070c57f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:33.706773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:33.786030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 676/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:34.064182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:34.066393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:34.167916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908ac66-3642-478d-be65-07a24e4da478 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:34.245797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:34.246428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:34.246717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:34.247018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:34.249357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:34.298783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:34.366902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:34.368343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80931e-ccaa-4d9a-be3e-bddead5390b8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:34.387906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 670/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:38.519315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:38.519884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:38.519884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:38.521265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 677/1347] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:06:38.526735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6b69-7297-4050-b062-e381b8dd350a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:38.527211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6b69-7297-4050-b062-e381b8dd350a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:38.527360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6b69-7297-4050-b062-e381b8dd350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:38.596250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a5eb6b69-7297-4050-b062-e381b8dd350a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:38.597559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5eb6b69-7297-4050-b062-e381b8dd350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:38.697192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:38.699847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 671/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:06:38.981046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:38.982645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:39.084685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e2f7e-7510-4b81-8d60-3d40b8bf2709 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:39.131131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:39.156902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:39.157525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:39.157803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:39.158071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:39.159636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:39.266718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:39.268121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39b45-e71a-454c-b048-6e24afb6039e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:39.286040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 678/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:06:44.146249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7eb7-1580-40e2-96dc-a59e988f19a0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:44.146249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7eb7-1580-40e2-96dc-a59e988f19a0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:44.146720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d7eb7-1580-40e2-96dc-a59e988f19a0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:44.147668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 672/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:44.159374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078398-6ae8-43ea-9a20-8fe2ebb3c04f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:44.159800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078398-6ae8-43ea-9a20-8fe2ebb3c04f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:44.159970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078398-6ae8-43ea-9a20-8fe2ebb3c04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:44.286745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-38078398-6ae8-43ea-9a20-8fe2ebb3c04f None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:44.288479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-38078398-6ae8-43ea-9a20-8fe2ebb3c04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:44.309598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:44.428509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 679/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:06:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:48.541262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c3c60-9558-4e42-9b5f-8cc6a3ac51bb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:48.541262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c3c60-9558-4e42-9b5f-8cc6a3ac51bb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:48.541262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c3c60-9558-4e42-9b5f-8cc6a3ac51bb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:48.542791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 673/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:48.548016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3cf5-7e78-44b0-9478-fb17b552be1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:48.548879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3cf5-7e78-44b0-9478-fb17b552be1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:48.550013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3cf5-7e78-44b0-9478-fb17b552be1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:48.634359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e01c3cf5-7e78-44b0-9478-fb17b552be1b None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:48.635974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e01c3cf5-7e78-44b0-9478-fb17b552be1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:48.720195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:48.768555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 680/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:06:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:50.145253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c89830-8bd3-4069-a39f-0624a30c6813 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:50.145826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c89830-8bd3-4069-a39f-0624a30c6813 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:50.145939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c89830-8bd3-4069-a39f-0624a30c6813 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:50.147353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 674/1354] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:50.152478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde927-82e6-41e2-895a-fb6bd37ccbbf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:50.152917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde927-82e6-41e2-895a-fb6bd37ccbbf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:50.153021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecde927-82e6-41e2-895a-fb6bd37ccbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:50.228249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4ecde927-82e6-41e2-895a-fb6bd37ccbbf None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:50.229458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ecde927-82e6-41e2-895a-fb6bd37ccbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:50.338345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 681/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:06:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:06:53.647433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3bca-1aef-4c15-a5c3-687d34bcde31 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:53.647985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3bca-1aef-4c15-a5c3-687d34bcde31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:53.647985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee3bca-1aef-4c15-a5c3-687d34bcde31 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:53.649448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 675/1356] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:53.654585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ffd2-9304-4b13-8d73-bc04b2a2f915 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:53.654985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ffd2-9304-4b13-8d73-bc04b2a2f915 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:53.655207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ffd2-9304-4b13-8d73-bc04b2a2f915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:53.695042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:53.743481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d695ffd2-9304-4b13-8d73-bc04b2a2f915 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:53.744682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d695ffd2-9304-4b13-8d73-bc04b2a2f915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:53.878025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 682/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:54.172405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:54.172843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:54.173296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:54.175594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 676/1358] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 07:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:54.185059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801d173-61c3-43e1-8f8f-81a262baf215 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:54.185673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801d173-61c3-43e1-8f8f-81a262baf215 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:54.186052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801d173-61c3-43e1-8f8f-81a262baf215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:54.273984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a801d173-61c3-43e1-8f8f-81a262baf215 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:54.275419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a801d173-61c3-43e1-8f8f-81a262baf215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:54.325774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:54.404980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 683/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 07:06:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:06:54.414293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:54.415961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:54.574947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cc543-63a6-411d-96cc-d89dcd6c212b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:54.657083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:54.699149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:54.699936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:54.700311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:54.700687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:54.702418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:54.804175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:54.805946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c5082-0e00-4baf-834f-2a770d68fbea service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:54.833304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 677/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:06:56.958682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:56.959303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:56.959303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:56.960749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 684/1361] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:06:56.965841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8838e-ae7e-4a49-b491-3f17c47838ee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:56.966535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8838e-ae7e-4a49-b491-3f17c47838ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:56.966535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8838e-ae7e-4a49-b491-3f17c47838ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:57.034222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-35a8838e-ae7e-4a49-b491-3f17c47838ee None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:06:57.035409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35a8838e-ae7e-4a49-b491-3f17c47838ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:57.073358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:57.141380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 678/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:06:57.424351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:06:57.426252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:57.606208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87bb76-a447-4aa4-bc75-8990a8dd53d1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:06:57.616226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:06:57.725447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:06:57.725950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:06:57.726226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:06:57.726507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:06:57.728093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:06:57.836636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:06:57.838361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d322b5-da35-4141-8e9d-9b9f8e839fa5 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:06:57.857443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 685/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:00.104055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aa6cb-d109-4842-b697-0c2cf5884634 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:00.104591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aa6cb-d109-4842-b697-0c2cf5884634 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:00.104591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aa6cb-d109-4842-b697-0c2cf5884634 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:00.106152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 679/1364] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:00.111678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5605db-23eb-46a3-90e5-d9f562c8b575 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:00.112542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5605db-23eb-46a3-90e5-d9f562c8b575 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:00.113016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5605db-23eb-46a3-90e5-d9f562c8b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:00.132531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:00.204509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8a5605db-23eb-46a3-90e5-d9f562c8b575 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:00.206499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8a5605db-23eb-46a3-90e5-d9f562c8b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:00.325430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 686/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:00.619370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:00.619669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:00.619915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:00.621424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 680/1366] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 07:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:00.627849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ca00-4af4-4714-8c70-6c5df0138c93 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:00.628288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ca00-4af4-4714-8c70-6c5df0138c93 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:00.628707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ca00-4af4-4714-8c70-6c5df0138c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:00.707920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ffb2ca00-4af4-4714-8c70-6c5df0138c93 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:00.709346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ffb2ca00-4af4-4714-8c70-6c5df0138c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:00.866000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:00.955397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 687/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 07:07:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:00.963775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:00.965323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:01.121898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b726f7-fe76-444f-8cbb-a4cee044cb76 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:01.144020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:01.243892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:01.244512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:01.244792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:01.245071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:01.246839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:01.345004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:01.347160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c065676-1feb-41ab-943a-dc11509650f8 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:01.366714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 681/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:01.865465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b546a-6939-4964-bbfd-b6194562c6a9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:01.865991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b546a-6939-4964-bbfd-b6194562c6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:01.865991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b546a-6939-4964-bbfd-b6194562c6a9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:01.867521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 688/1369] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:01.872563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477329-76d2-4c3b-9a54-80ee063a5235 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:01.872846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477329-76d2-4c3b-9a54-80ee063a5235 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:01.873133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c477329-76d2-4c3b-9a54-80ee063a5235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:01.907691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:01.946523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c477329-76d2-4c3b-9a54-80ee063a5235 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:01.947726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c477329-76d2-4c3b-9a54-80ee063a5235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:02.051845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 682/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:02.633683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:02.633683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:02.633683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:02.637156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 689/1371] 10.222.0.46 () {54 vars in 964 bytes} [Thu Sep 1 07:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:02.640287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d123d-6459-41a5-9af9-1b187876688d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:02.641080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d123d-6459-41a5-9af9-1b187876688d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:02.641574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d123d-6459-41a5-9af9-1b187876688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:02.719753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2f9d123d-6459-41a5-9af9-1b187876688d None None] MFA Rules not processed for user `92e78fff3061449ba36aafbc1b3f7eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:02.721311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f9d123d-6459-41a5-9af9-1b187876688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:02.778796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:02.811293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 683/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Sep 1 07:07:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:02.819559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:02.821261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:02.928276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13ce2f-b37d-408a-9cac-c63167340cde None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:02.970499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:03.011133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:03.011745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:03.012026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:03.012303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:03.014160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:03.133951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:03.135371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02dc8a-f556-4e47-b9a1-29de22570c3c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:03.156198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 690/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:03.349030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:03.349348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:03.349629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:03.350990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 684/1374] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:03.356198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1181-2d80-4558-926c-8d167d1653c2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:03.356490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1181-2d80-4558-926c-8d167d1653c2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:03.356792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1181-2d80-4558-926c-8d167d1653c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:03.429650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-72cf1181-2d80-4558-926c-8d167d1653c2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:03.430890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-72cf1181-2d80-4558-926c-8d167d1653c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:03.450950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:03.526028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 691/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:03.816321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:03.817893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:03.966450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4767-e787-492e-a744-7c3749a1c413 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:04.017222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:04.081288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:04.082021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:04.082299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:04.082574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:04.084164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:04.178933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:04.180554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52061ba4-7702-4603-9049-7b51a199d4d6 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:04.198410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 685/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:06.135125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:06.135582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:06.135582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:06.136938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 692/1377] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:06.142079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bb2b-412a-4294-8f62-931f86c3767d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:06.142490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bb2b-412a-4294-8f62-931f86c3767d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:06.142700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96bb2b-412a-4294-8f62-931f86c3767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:06.212958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3c96bb2b-412a-4294-8f62-931f86c3767d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:06.214388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c96bb2b-412a-4294-8f62-931f86c3767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:06.225469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:06.330148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 686/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:06.623021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:06.624788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:06.820820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4793a2-b9be-4f55-bbf0-c467fa76bed5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:06.824921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:06.945615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:06.946132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:06.946403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:06.946668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:06.948876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:07.042214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:07.043796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae3911-31c5-4ea6-a2fd-254161afd1e3 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:07.062701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 693/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:09.015342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:09.015861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:09.015861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:09.017456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 687/1380] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:09.023001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab3df-bd7a-4986-8fda-1d14feac3c14 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:09.023442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab3df-bd7a-4986-8fda-1d14feac3c14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:09.023610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cab3df-bd7a-4986-8fda-1d14feac3c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:09.039503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:09.097142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b7cab3df-bd7a-4986-8fda-1d14feac3c14 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:09.098744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b7cab3df-bd7a-4986-8fda-1d14feac3c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:09.206840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 694/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:09.500042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:09.501718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:09.662720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:09.674062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf9d80-b8a2-4e53-9e0a-566549900b20 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:09.800993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:09.801632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:09.801908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:09.802173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:09.803761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:09.902824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:09.904433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e332539-68b1-4b4a-a2bb-13c12a64e797 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:09.915895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:09.925619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 688/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:12.177170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:12.177702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:12.177702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:12.179162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 695/1383] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:07:12.185261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b037d-11a8-4f81-b1a6-fb2f6a8435a8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:12.185735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b037d-11a8-4f81-b1a6-fb2f6a8435a8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:12.185948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b037d-11a8-4f81-b1a6-fb2f6a8435a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:12.268072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba2b037d-11a8-4f81-b1a6-fb2f6a8435a8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:12.269407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba2b037d-11a8-4f81-b1a6-fb2f6a8435a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:12.378569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 689/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:12.700790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:12.702355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:12.791837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:12.854899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3924e-60f4-4374-917a-f8c79731d4ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:12.986000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:12.987070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:12.987343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:12.987609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:12.989279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:13.058391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:13.099874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:13.101490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbcf90-6dea-4e95-ba3f-8a7b7501c8cf service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:13.119650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 696/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:15.025947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:15.026475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:15.026575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:15.027889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 690/1386] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:07:15.033949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c9b0f-9f8c-4070-b76f-37a2f4a056ab None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:15.034536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c9b0f-9f8c-4070-b76f-37a2f4a056ab None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:15.034637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c9b0f-9f8c-4070-b76f-37a2f4a056ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:15.102471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-268c9b0f-9f8c-4070-b76f-37a2f4a056ab None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:15.103690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-268c9b0f-9f8c-4070-b76f-37a2f4a056ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:15.204644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 697/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:07:15.494513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:15.496786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:15.576956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:15.655283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ac3b2-dab6-47cd-b53c-d77a81c8d274 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:15.799134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:15.800157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:15.800777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:15.801207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:15.803172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:15.844026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:15.917365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:15.919682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e150f-46db-44b9-a5b8-0d7934de2fcb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:15.944524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 691/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:17.911449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3fdb-1896-4c7f-aa67-ab3c4ec8fd75 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:17.912015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3fdb-1896-4c7f-aa67-ab3c4ec8fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:17.912015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf3fdb-1896-4c7f-aa67-ab3c4ec8fd75 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:17.913393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 698/1389] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:17.918936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2b04-3235-4270-bab2-65c746d5a191 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:17.919391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2b04-3235-4270-bab2-65c746d5a191 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:17.919485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2b04-3235-4270-bab2-65c746d5a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:17.985073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-135d2b04-3235-4270-bab2-65c746d5a191 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:17.986303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-135d2b04-3235-4270-bab2-65c746d5a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.085598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 692/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:18.140373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.140656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.140936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.142320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 699/1391] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:18.148118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935de4c-2a15-4593-a0b9-4f0fc049499d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.148405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935de4c-2a15-4593-a0b9-4f0fc049499d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.148682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935de4c-2a15-4593-a0b9-4f0fc049499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.164262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:18.215049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f935de4c-2a15-4593-a0b9-4f0fc049499d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:18.216342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f935de4c-2a15-4593-a0b9-4f0fc049499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.319739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 693/1392] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:18.376398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:18.378569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.409019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:18.524571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543d2c0-0572-465a-8395-1cd1d9d1525b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:18.619570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3e7e-e318-4d05-9e34-4ec66af528ee None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.619861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3e7e-e318-4d05-9e34-4ec66af528ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.620156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf3e7e-e318-4d05-9e34-4ec66af528ee None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.621519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 694/1393] 10.222.0.46 () {54 vars in 967 bytes} [Thu Sep 1 07:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:18.626361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76c295-8fd3-4223-9235-680e865e1253 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.626515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:18.626644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76c295-8fd3-4223-9235-680e865e1253 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.626921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76c295-8fd3-4223-9235-680e865e1253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.638824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:18.639421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.639701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.639968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.641609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.703285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1e76c295-8fd3-4223-9235-680e865e1253 None None] MFA Rules not processed for user `9a9b0f324af44c0f8827b31befa6939b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:18.704498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e76c295-8fd3-4223-9235-680e865e1253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.731720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:18.733349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:18.744758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:18.752574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 700/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:18.792729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 695/1395] 10.222.0.46 () {58 vars in 1064 bytes} [Thu Sep 1 07:07:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:07:18.800953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:18.802294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:18.885082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06389e5-2971-47c5-909e-cbdacacf2067 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:18.929531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:18.963947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:18.964546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:18.964821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:18.965089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:18.966689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:19.075443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:19.076891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b5132-a85d-4c3b-87b9-d7a028910958 service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:19.095316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 701/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:19.895217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c257-4d21-457c-8368-6ed4df77841b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:19.895791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c257-4d21-457c-8368-6ed4df77841b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:19.895791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917c257-4d21-457c-8368-6ed4df77841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:19.968173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8917c257-4d21-457c-8368-6ed4df77841b None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:19.969396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8917c257-4d21-457c-8368-6ed4df77841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:20.059962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 696/1397] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:07:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:20.102773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:20.103261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:20.103416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:20.104753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 702/1398] 10.222.0.46 () {54 vars in 943 bytes} [Thu Sep 1 07:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:20.110841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dab6d-4701-4ff0-8c35-0246e249867f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:20.111767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dab6d-4701-4ff0-8c35-0246e249867f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:20.112052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dab6d-4701-4ff0-8c35-0246e249867f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:20.170005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:20.189542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-411dab6d-4701-4ff0-8c35-0246e249867f None None] MFA Rules not processed for user `6834988c33e94b9cb41df95e4bb1bd88`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:20.190910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-411dab6d-4701-4ff0-8c35-0246e249867f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:20.331400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 697/1399] 10.222.0.46 () {58 vars in 1040 bytes} [Thu Sep 1 07:07:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:20.342667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:20.344328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:20.451459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:20.520350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebd2d5-aaf9-4f46-b8cc-61257f3e1d41 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:20.623578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:20.624373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:20.624641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:20.624913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:20.626560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:20.684545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:20.709927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:20.711514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a87819-c5bf-4f9f-9319-6e23a222234e service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:20.719713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 703/1400] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:07:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:07:21.240940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:21.240940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:21.241654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:21.242981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 698/1401] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:21.248763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a8d7-d373-40df-8f3a-2653d69a131b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:21.249559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a8d7-d373-40df-8f3a-2653d69a131b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:21.249655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a8d7-d373-40df-8f3a-2653d69a131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:21.356043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6255a8d7-d373-40df-8f3a-2653d69a131b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:21.357301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6255a8d7-d373-40df-8f3a-2653d69a131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:21.467278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 704/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:21.759980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:21.764872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:21.886349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:21.938818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7b424-cd72-466e-97c8-f2093e166b69 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:22.071490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:22.072097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:22.072696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:22.072976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:22.074588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:22.148724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:22.169429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:22.171148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe270206-45e2-46f1-8b8d-dbd6414244b8 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:22.194895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 699/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:24.150448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:24.151043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:24.151043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:24.152510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 705/1404] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:24.158317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab3166-c1f2-418e-a3b8-4d5871a3809d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:24.159343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab3166-c1f2-418e-a3b8-4d5871a3809d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:24.160554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ab3166-c1f2-418e-a3b8-4d5871a3809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:24.242693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8ab3166-c1f2-418e-a3b8-4d5871a3809d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:24.243909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8ab3166-c1f2-418e-a3b8-4d5871a3809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:24.393019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 700/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:24.686357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:24.687871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:24.740068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:24.856779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874b133-fee1-44e1-94a2-6e81784ef600 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:24.979656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:24.980165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:24.980441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:24.980710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:24.982411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:24.994934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:25.084772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:25.086421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fe81e-f0cb-4f51-b571-fc509ca1856c service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:25.105072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 706/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:27.056089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:27.056671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:27.056671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:27.058204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 701/1407] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:27.063287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fcc50-956d-44e5-aab7-2b87d184ec70 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:27.063718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fcc50-956d-44e5-aab7-2b87d184ec70 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:27.063829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fcc50-956d-44e5-aab7-2b87d184ec70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:27.134723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-918fcc50-956d-44e5-aab7-2b87d184ec70 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:27.136022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-918fcc50-956d-44e5-aab7-2b87d184ec70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:27.235253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:27.250253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 707/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:27.547266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:27.548861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:27.604300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:27.719597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634c113-af18-43e3-a357-d36b8e345ba4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:27.848516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:27.849495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:27.849978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:27.850252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:27.851818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:27.861065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:27.951866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:27.953601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68d923-22a3-4f17-ad39-1ab7c736ba82 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:27.972705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 702/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:37.248263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:37.969894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=41 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:42.504018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fda7-8f01-4d12-baac-4127d8f6592b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:42.504551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fda7-8f01-4d12-baac-4127d8f6592b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:42.504551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74fda7-8f01-4d12-baac-4127d8f6592b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:42.505882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 708/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu Sep 1 07:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:42.510860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce501f6-656b-4d84-85ea-81f5830a20e8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:42.511286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce501f6-656b-4d84-85ea-81f5830a20e8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:42.511400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce501f6-656b-4d84-85ea-81f5830a20e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:42.586512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2ce501f6-656b-4d84-85ea-81f5830a20e8 None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:42.587709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ce501f6-656b-4d84-85ea-81f5830a20e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:42.693312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 703/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:42.707854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89ec1c-55bd-4203-8f52-e50c865a64f4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:42.708143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89ec1c-55bd-4203-8f52-e50c865a64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:42.708422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89ec1c-55bd-4203-8f52-e50c865a64f4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:42.709809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 709/1412] 10.222.0.46 () {54 vars in 918 bytes} [Thu Sep 1 07:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:42.714229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967620c6-e43f-4ae6-b70c-5f8b65eacd8b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:42.714542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967620c6-e43f-4ae6-b70c-5f8b65eacd8b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:42.714814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967620c6-e43f-4ae6-b70c-5f8b65eacd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:42.788395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:42.791510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-967620c6-e43f-4ae6-b70c-5f8b65eacd8b None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:42.793058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-967620c6-e43f-4ae6-b70c-5f8b65eacd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:42.905763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 704/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Sep 1 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:45.244533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:45.245071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:45.245071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:45.246422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:45.251914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfbdf9-30c6-4e87-a15e-e5ad82ed53bb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:45.252358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfbdf9-30c6-4e87-a15e-e5ad82ed53bb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:45.252455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfbdf9-30c6-4e87-a15e-e5ad82ed53bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:45.319760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1dcfbdf9-30c6-4e87-a15e-e5ad82ed53bb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:45.321001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1dcfbdf9-30c6-4e87-a15e-e5ad82ed53bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:45.375803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:45.418397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:45.714273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:45.715799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:45.860765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9528c5-a158-4177-b4b8-4e90bd4c3593 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:45.931585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:45.974185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:45.974710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:45.974987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:45.975256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:45.976856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:46.065029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:46.066623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc6053-e161-48bd-b0af-f7d2ac323fa0 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:46.084203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 711/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:46.966877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:46.967434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:46.967434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:46.968768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:46.974327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e04c9-6454-428c-81e3-f784bf740f2e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:46.974792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e04c9-6454-428c-81e3-f784bf740f2e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:46.974909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e04c9-6454-428c-81e3-f784bf740f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:47.050973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b15e04c9-6454-428c-81e3-f784bf740f2e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:47.052192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b15e04c9-6454-428c-81e3-f784bf740f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:47.078323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:47.148587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:47.297471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:47.299010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:47.442850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ebe0b-497d-4066-8ff3-9d4b40a44989 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:47.481733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:47.555979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:47.556584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:47.556867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:47.557157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:47.558749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:47.645641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:47.647234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c52807-9662-4ede-9052-4a78aa87dd81 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:47.665081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 707/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:48.540815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:48.541423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:48.541524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:48.542823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:48.547849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ad397-0100-4e1a-97ed-bfac94fe07da None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:48.548256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ad397-0100-4e1a-97ed-bfac94fe07da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:48.548392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ad397-0100-4e1a-97ed-bfac94fe07da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:48.601357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:48.622594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e94ad397-0100-4e1a-97ed-bfac94fe07da None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:48.623781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e94ad397-0100-4e1a-97ed-bfac94fe07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:48.717413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:48.865947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:48.867425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:49.010475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738569b-3c7d-4ff4-b9d6-516947c50e66 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:49.023226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:49.121169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:49.121783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:49.122061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:49.122330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:49.123904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:49.209825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:49.211392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03738e35-4305-4406-bdde-f9ffcad4c379 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:49.229375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 714/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:50.080030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:50.080434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:50.080563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:50.081945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:50.086729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227391-4c17-4509-84c2-34597829057b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:50.087171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227391-4c17-4509-84c2-34597829057b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:50.087268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227391-4c17-4509-84c2-34597829057b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:50.107330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:50.163855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f6227391-4c17-4509-84c2-34597829057b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:50.165041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f6227391-4c17-4509-84c2-34597829057b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:50.263316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:50.411480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:50.412932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:50.520831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:50.556560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abced47-fe02-45e5-8212-92f571148c08 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:50.673176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:50.674044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:50.674318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:50.674581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:50.676977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:50.757960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:50.773662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:50.776656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590646c3-8901-4c1e-9854-a83f27611138 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:50.797339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 710/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:51.670823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:51.671398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:51.671398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:51.672709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:51.677755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ff5f-06a7-478f-8949-e5ba581c3d18 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:51.678162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ff5f-06a7-478f-8949-e5ba581c3d18 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:51.678304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850ff5f-06a7-478f-8949-e5ba581c3d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:51.754124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3850ff5f-06a7-478f-8949-e5ba581c3d18 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:51.755337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3850ff5f-06a7-478f-8949-e5ba581c3d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:51.850331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:52.006916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:52.008337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:52.091390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:52.152405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940e549-c851-4fc7-b315-e415f5d5894b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:52.262921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:52.263451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:52.263728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:52.263995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:52.265613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:52.313344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:52.357625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:52.359997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c3bbb-94dd-44c1-9e32-3da1d8604807 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:52.379064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 717/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:53.180264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:53.180834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:53.180834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:53.182225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:53.187210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db814e-d63c-4d60-af10-2f727a218a06 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:53.187638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db814e-d63c-4d60-af10-2f727a218a06 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:53.187747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db814e-d63c-4d60-af10-2f727a218a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:53.264063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2db814e-d63c-4d60-af10-2f727a218a06 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:53.265305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2db814e-d63c-4d60-af10-2f727a218a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:53.361473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:53.513246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:53.514729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:53.562690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:53.665793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3dcf-e1a3-4d2f-8338-19a1094a41be None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:53.780056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:53.780656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:53.780928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:53.781218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:53.783429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:53.790183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:53.870314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:53.871939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150272a-2e0e-4dc4-9ec3-66a598ff244e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:53.890292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 713/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:54.687545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:54.688127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:54.688127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:54.689997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:54.694775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793ea0-933d-4f77-ba3c-d9ad08fccd62 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:54.695227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793ea0-933d-4f77-ba3c-d9ad08fccd62 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:54.695332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793ea0-933d-4f77-ba3c-d9ad08fccd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:54.771383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44793ea0-933d-4f77-ba3c-d9ad08fccd62 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:54.772587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44793ea0-933d-4f77-ba3c-d9ad08fccd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:54.839927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:54.870246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:55.026014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:55.027569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:55.047837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:55.177453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef5e8-9965-4a9b-820b-eeee3671f2b1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:55.262942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:55.288051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:55.288647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:55.288922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:55.289217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:55.290791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:55.387255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:55.388841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be3a96-e30f-436e-b339-abc3210f782d service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:55.406326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 720/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:56.200713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:56.201299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:56.201299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:56.202628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:56.208147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe69d-f0a3-43fa-b00d-074a0f41b618 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:56.208543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe69d-f0a3-43fa-b00d-074a0f41b618 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:56.208697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe69d-f0a3-43fa-b00d-074a0f41b618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:56.286549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-544fe69d-f0a3-43fa-b00d-074a0f41b618 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:56.287766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-544fe69d-f0a3-43fa-b00d-074a0f41b618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:56.328184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:56.383800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:56.532795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:56.534358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:56.688605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797ff83-6a51-4257-896b-256bca7fbc6c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:56.743162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:56.807162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:56.807816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:56.808093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:56.808359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:56.809960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:56.898045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:56.899627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53087e-3f78-4ad8-995d-642303624e8c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:56.917816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 716/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:07:57.726712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:57.727298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:57.727298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:57.728598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:07:57.733570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4248-72c1-4e02-9528-7247ef2a1ed4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:57.733979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4248-72c1-4e02-9528-7247ef2a1ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:57.734119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac4248-72c1-4e02-9528-7247ef2a1ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:57.809843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-72ac4248-72c1-4e02-9528-7247ef2a1ed4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:07:57.811040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-72ac4248-72c1-4e02-9528-7247ef2a1ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:57.813032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:57.910117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:07:58.058761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:07:58.060282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:58.202612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2a97-d2c3-4e71-a026-34338eb13509 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:07:58.231151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:07:58.321860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:07:58.322370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:07:58.322643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:07:58.322911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:07:58.324939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:07:58.414828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:07:58.416403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b8674-ec1d-46e2-abd5-09cfe62e2108 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:07:58.434361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 723/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:00.243824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:00.244372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:00.244372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:00.245754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:00.251201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5126a7-3b05-4d30-ba8e-4f98dbe223b0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:00.251629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5126a7-3b05-4d30-ba8e-4f98dbe223b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:00.251760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5126a7-3b05-4d30-ba8e-4f98dbe223b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:00.296094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:00.325769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5b5126a7-3b05-4d30-ba8e-4f98dbe223b0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:00.327061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5b5126a7-3b05-4d30-ba8e-4f98dbe223b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:00.429524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:00.720873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:00.722503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:00.824333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747abb-228e-4b8e-8574-ff508ac2f955 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:00.855838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:00.897100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:00.897732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:00.898008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:00.898278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:00.899860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:01.010410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:01.011758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fe152-2e18-4b0d-974b-ca5e9cf59076 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:01.037274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 719/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:03.059604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:03.060202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:03.060202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:03.061516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:03.066752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e563f0-f29a-4486-89de-7c8ae17064bd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:03.067166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e563f0-f29a-4486-89de-7c8ae17064bd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:03.067326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e563f0-f29a-4486-89de-7c8ae17064bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:03.133706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-06e563f0-f29a-4486-89de-7c8ae17064bd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:03.134910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06e563f0-f29a-4486-89de-7c8ae17064bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:03.147687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:03.232131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:03.515094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:03.516628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:03.618879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75148b-c5b4-4470-bdf6-6bf1b35d01ff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:03.631393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:03.692451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:03.693043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:03.693348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:03.693618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:03.695187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:03.801332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:03.802770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d013e7-c85e-463d-b44d-f77c68b1d5c3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:03.821200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 726/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:06.029543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:06.030080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:06.030179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:06.031548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:06.036634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3592e-d22b-439e-a147-df8802d2a630 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:06.037031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3592e-d22b-439e-a147-df8802d2a630 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:06.037215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3592e-d22b-439e-a147-df8802d2a630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:06.085368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:06.103219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c9f3592e-d22b-439e-a147-df8802d2a630 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:06.104420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c9f3592e-d22b-439e-a147-df8802d2a630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:06.199953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:06.498649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:06.500453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:06.601198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2822-dc70-44c7-bf3e-3f6a86924c9d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:06.640078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:06.673231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:06.673823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:06.674125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:06.674395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:06.675948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:06.783272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:06.784751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc24a-8da0-434c-97b2-25791a36585f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:06.802994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 722/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:08.830166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:08.830782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:08.830782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:08.832097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:08.837288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997399d7-d665-44e9-a73d-9d210617e35c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:08.837717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997399d7-d665-44e9-a73d-9d210617e35c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:08.837863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997399d7-d665-44e9-a73d-9d210617e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:08.904055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-997399d7-d665-44e9-a73d-9d210617e35c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:08.905287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-997399d7-d665-44e9-a73d-9d210617e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:08.926440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:09.001913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:08:09.298080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:09.299646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:09.409193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbdde2b-7ab5-4d43-8dc1-c152731e1a2a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:09.429365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:09.483375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:09.483975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:09.484252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:09.484522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:09.486149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:09.591344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:09.592696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def2e72-2290-47d7-934e-9a3194c2bc5f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:09.610907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 729/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:11.560204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:11.560767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:11.560767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:11.562180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:08:11.567541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7feef-3f5d-4cf6-a121-f2bfa523ba0f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:11.567965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7feef-3f5d-4cf6-a121-f2bfa523ba0f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:11.568085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7feef-3f5d-4cf6-a121-f2bfa523ba0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:11.631193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:11.634634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-32e7feef-3f5d-4cf6-a121-f2bfa523ba0f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:11.636043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32e7feef-3f5d-4cf6-a121-f2bfa523ba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:11.730549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:12.012400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:12.013981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:12.116792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf419a8-ee09-47df-b4f6-16cbc0e5ace7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:12.164425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:12.189490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:12.190119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:12.190391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:12.190659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:12.192264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:12.298941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:12.300369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0707115-1149-4351-a285-2ea3ee04df22 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:12.317704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 725/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:14.292992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:14.293589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:14.293589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:14.294915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:14.301034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207408-d6b2-449e-8bff-2c58aed4a0a8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:14.301463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207408-d6b2-449e-8bff-2c58aed4a0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:14.301604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207408-d6b2-449e-8bff-2c58aed4a0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:14.367097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a3207408-d6b2-449e-8bff-2c58aed4a0a8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:14.368209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3207408-d6b2-449e-8bff-2c58aed4a0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:14.407738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:14.476507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:14.766296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:14.768371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:14.871713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1ef4-745a-4238-99ac-8f37724ebaaa None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:14.900845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:14.946017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:14.946524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:14.946799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:14.947093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:14.948655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:15.053304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:15.054715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14c7e2-f66f-45d8-915a-8622e8d238de service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:15.072826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 732/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:17.033027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:17.034060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:17.034060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:17.035065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:17.040345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c53-4fb2-4079-8264-564412c2c019 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:17.040749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c53-4fb2-4079-8264-564412c2c019 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:17.040887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c16c53-4fb2-4079-8264-564412c2c019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:17.105807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-71c16c53-4fb2-4079-8264-564412c2c019 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:17.106996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-71c16c53-4fb2-4079-8264-564412c2c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:17.115701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:17.202715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:17.489013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:17.490602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:17.597025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63361a5-23c6-4eee-83e6-0fd62775e069 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:17.651142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:17.669963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:17.670570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:17.670849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:17.671120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:17.672717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:17.780925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:17.782392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574878d1-05da-4b79-bb6b-ff664499feb5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:17.800443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 728/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:19.741551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:19.742138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:19.742138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:19.743452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:19.748713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679199-8cdf-4db0-a41d-c78697a6743f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:19.749166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679199-8cdf-4db0-a41d-c78697a6743f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:19.749283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679199-8cdf-4db0-a41d-c78697a6743f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:19.820891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b679199-8cdf-4db0-a41d-c78697a6743f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:19.822131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b679199-8cdf-4db0-a41d-c78697a6743f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:19.859025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:19.919499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:20.197786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:20.199329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:20.302243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1800b4-8706-4951-8b84-af0d52023462 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:20.339807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:20.375995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fa641-b314-4473-a548-d580b276505d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:20.376523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa641-b314-4473-a548-d580b276505d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:20.376803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa641-b314-4473-a548-d580b276505d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:20.377108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fa641-b314-4473-a548-d580b276505d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:20.378739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-796fa641-b314-4473-a548-d580b276505d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:20.488795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fa641-b314-4473-a548-d580b276505d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:20.490270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fa641-b314-4473-a548-d580b276505d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:20.508116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 735/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:22.424612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:22.425177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:22.425177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:22.426485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:22.431623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192f28-99b3-409a-9349-f2f307bb1885 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:22.432056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192f28-99b3-409a-9349-f2f307bb1885 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:22.432199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192f28-99b3-409a-9349-f2f307bb1885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:22.497017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c4192f28-99b3-409a-9349-f2f307bb1885 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:22.498246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c4192f28-99b3-409a-9349-f2f307bb1885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:22.516535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:22.593296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:22.869229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:22.870796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:22.974359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd6a65-2bde-4ed7-b24c-f739dfea97b0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:23.039586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:23.046654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:23.047262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:23.047542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:23.047816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:23.049458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:23.155950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:23.157682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dd1f8-e029-4b9b-8d4f-75c906ff3f96 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:23.175279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 731/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:25.109671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:25.110261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:25.110261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:25.111578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:25.116728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bf0-c0d7-447d-8b3c-d143d3fef298 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:25.117199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bf0-c0d7-447d-8b3c-d143d3fef298 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:25.117314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bf0-c0d7-447d-8b3c-d143d3fef298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:25.182282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0bb92bf0-c0d7-447d-8b3c-d143d3fef298 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:25.183705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bb92bf0-c0d7-447d-8b3c-d143d3fef298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:25.232730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:25.281711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:08:25.566225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:25.567757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:25.671067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df9d10-1cfc-4550-9384-6f76b70fed46 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:25.721765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:25.748022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:25.748628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:25.748908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:25.749239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:25.751103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:25.862968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:25.864318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07e5eb-9072-48fa-9fe3-7243220a38d2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:25.882465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 738/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:35.279858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=22 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:35.880220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=66 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:45.282201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936da8-fb51-4445-8d74-87647f061cdc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:45.282780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936da8-fb51-4445-8d74-87647f061cdc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:45.282780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8936da8-fb51-4445-8d74-87647f061cdc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:45.284286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 733/1471] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:45.289620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183716ae-db3d-4612-8577-55aa017e1ce1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:45.290093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183716ae-db3d-4612-8577-55aa017e1ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:45.290203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183716ae-db3d-4612-8577-55aa017e1ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:45.357873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-183716ae-db3d-4612-8577-55aa017e1ce1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:45.359092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-183716ae-db3d-4612-8577-55aa017e1ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:45.453923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 739/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:45.753777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:45.754064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:45.754338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:45.755679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 734/1473] 10.222.0.46 () {54 vars in 967 bytes} [Thu Sep 1 07:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:45.760557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3406e-5e4c-4b8b-b325-d5f55fd226a1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:45.760918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3406e-5e4c-4b8b-b325-d5f55fd226a1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:45.761301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3406e-5e4c-4b8b-b325-d5f55fd226a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:45.835163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c4d3406e-5e4c-4b8b-b325-d5f55fd226a1 None None] MFA Rules not processed for user `9a9b0f324af44c0f8827b31befa6939b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:45.836364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c4d3406e-5e4c-4b8b-b325-d5f55fd226a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:45.857477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:45.912978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 740/1474] 10.222.0.46 () {58 vars in 1064 bytes} [Thu Sep 1 07:08:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:45.920624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:45.922204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:46.023939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4709a-09ff-4742-b9d9-4b343a50e0e2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:46.096753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:46.097366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:46.097639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:46.097907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:46.099465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:46.151221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:46.205879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:46.207319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3246b-d7dc-4793-83a7-3d863d4d574b service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:46.226453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 735/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:48.208487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:48.209016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:48.209016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:48.210398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 741/1476] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:48.215791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f37f4-401f-4276-8ec0-4d2a93e5f02a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:48.216240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f37f4-401f-4276-8ec0-4d2a93e5f02a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:48.216338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f37f4-401f-4276-8ec0-4d2a93e5f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:48.282200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a17f37f4-401f-4276-8ec0-4d2a93e5f02a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:48.283400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a17f37f4-401f-4276-8ec0-4d2a93e5f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:48.378017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 736/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:48.651743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:48.653314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:48.781074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e7423-136d-4de0-bfa1-2f19a4f5d6c7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:48.828977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:48.872769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:48.873391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:48.873662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:48.873934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:48.874560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:48.876506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:08:48.876506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:08:48.876506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:08:48.877676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109a96e-4677-4aa4-b1ac-c3862cf5377a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:48.895774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 742/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Sep 1 07:08:48 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:08:50.816830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:50.817341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:50.817453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:50.818769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 737/1479] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:50.823869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431f19-aca0-4805-ab25-6c433c8228cf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:50.824301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431f19-aca0-4805-ab25-6c433c8228cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:50.824409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f431f19-aca0-4805-ab25-6c433c8228cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:50.889807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5f431f19-aca0-4805-ab25-6c433c8228cf None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:50.891013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5f431f19-aca0-4805-ab25-6c433c8228cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:50.984738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 743/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:51.278507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:51.280098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:51.289708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:51.425265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963a7dd-a3cb-4095-8147-249216ca26d1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:51.507939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:51.542525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:51.543135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:51.543410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:51.543676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:51.545291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:51.638998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:51.640745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc9834-b6bc-493f-88cd-081902af6b85 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:51.662422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 738/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:53.782314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:53.782839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:53.782839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:53.784230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 744/1482] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:53.789347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7bff-1346-4971-8054-1cef7d74d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:53.789744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7bff-1346-4971-8054-1cef7d74d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:53.789901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e7bff-1346-4971-8054-1cef7d74d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:53.855117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-161e7bff-1346-4971-8054-1cef7d74d8f2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:53.856316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-161e7bff-1346-4971-8054-1cef7d74d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:53.902487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:53.952939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 739/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:54.239206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:54.240735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:54.264891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:54.347586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70bb48-59fc-4f88-adde-9617eae35a0e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:54.423636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:54.424271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:54.424554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:54.424827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:54.426466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:54.499755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:54.535186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:54.536618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7e711-2b3c-4aa3-a445-17b1652ccf01 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:54.554626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 745/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:56.524273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:56.524878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:56.524878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:56.526218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 740/1485] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:56.531566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1897-7b3f-4154-9360-9c3c46fa20f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:56.531994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1897-7b3f-4154-9360-9c3c46fa20f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:56.532121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1897-7b3f-4154-9360-9c3c46fa20f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:56.596863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8abd1897-7b3f-4154-9360-9c3c46fa20f5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:56.598098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8abd1897-7b3f-4154-9360-9c3c46fa20f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:56.694058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 746/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:56.985678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:56.987262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:57.097027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db85ba2-a8da-4420-a68e-c36362225a73 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:57.169994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:57.171764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:57.172564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:57.172996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:57.173456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:57.175040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:57.283559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:08:57.284990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf625c-e54c-4216-a07a-08393d34f93d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:08:57.302959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 741/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:08:59.280392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:59.280956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:59.280956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:59.282367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 747/1488] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:08:59.287933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f633f6-13a9-4bec-90f2-34e1309b4324 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:59.288344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f633f6-13a9-4bec-90f2-34e1309b4324 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:59.288476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f633f6-13a9-4bec-90f2-34e1309b4324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:59.352870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b0f633f6-13a9-4bec-90f2-34e1309b4324 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:08:59.354113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b0f633f6-13a9-4bec-90f2-34e1309b4324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:59.405574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:59.453016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 742/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:08:59.746058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:08:59.747607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:08:59.784893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:08:59.852086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688554-a995-4350-8d4e-8acb7a691103 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:08:59.925194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:08:59.925796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:08:59.926075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:08:59.926345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:08:59.927930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:09:00.007043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:09:00.035988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:09:00.037430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727320b-29a9-416e-8cf8-66fc2323fbf9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:09:00.055150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 748/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:09:02.265912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:09:02.266484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:09:02.266484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:09:02.267853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 743/1491] 10.222.0.46 () {54 vars in 927 bytes} [Thu Sep 1 07:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:09:02.273253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebfdc-e518-49e8-a346-28efbc8c4f4c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:09:02.273682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebfdc-e518-49e8-a346-28efbc8c4f4c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:09:02.273831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bebfdc-e518-49e8-a346-28efbc8c4f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:09:02.338616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-70bebfdc-e518-49e8-a346-28efbc8c4f4c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:09:02.339830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-70bebfdc-e518-49e8-a346-28efbc8c4f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:09:02.438295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 749/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Thu Sep 1 07:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:09:02.728006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:09:02.729585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:09:02.832360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549ed40-0131-4c5f-a13c-992045245bf8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:09:02.904179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:09:02.904800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:09:02.905057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:09:02.905351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:09:02.906924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:09:02.916023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:09:03.014654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:09:03.016008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf2ae7-4407-45a6-822e-9aaf1b439ce1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:09:03.034178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 744/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:09:12.436416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:09:13.031865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=50 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:36.475253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:36.475831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:36.475831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:36.549030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:10:36.550243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:36.644631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 750/1494] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:10:36.652467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:36.652944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:36.653048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:36.728533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:10:36.729709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:36.823887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 745/1495] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:10:36.828569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:36.829839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:36.841854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304218c-94cd-4520-93a3-0ac585926dc2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:36.902097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:36.936066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:36.936674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:36.936938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:36.937242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:36.937880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:10:36.938179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:36.939184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:36.939184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:10:36.939184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:10:36.940605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:36.951543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 751/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:10:36.960323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:36.969308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234be4c1-1434-416e-aac1-c73271ef3cb0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:37.028616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:37.061832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:37.062423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.062698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.062968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.063582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:37.065899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:37.065899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:10:37.065899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:10:37.067073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:37.121692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 746/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:10:36 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:10:37.126037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:37.128196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:37.162806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88568f-5d4b-4d8c-bb96-9832c63ed781 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:37.255777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:37.256373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.256652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.256924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.257582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:37.258581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:37.258581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:10:37.258581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:10:37.259796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:37.302135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 752/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:10:37 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:10:37.306251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:37.307760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:37.369783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb2654-4e1a-4226-9066-894cbcdb0db1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:37.451236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:37.462342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:37.462950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.463227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.463497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.464133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:37.465186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:37.465186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:10:37.465186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:10:37.466084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:37.478643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 747/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:10:37.482713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:37.484246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:37.496225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79720bf-b5e0-4202-9497-75b3153eb951 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:37.522085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:37.591153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:37.591669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.591949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.592222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.592862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:37.593901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:37.593901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:10:37.593901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:10:37.594792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b4b05-8414-426d-a01f-6af02eb92cd4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:37.607252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 753/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:10:37.612193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:37.613771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:37.648461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23e177-f191-40cb-92c4-aef040e8cbde admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:37.857445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:37.858103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.858270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.858534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] PATH_INFO: `/v3/projects/dd8cdd492b3e4529a2e972e6ebff9ade/users/0d4750061ec440e69712be3a830cc202/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.879468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8cdd492b3e4529a2e972e6ebff9ade, user_id=0d4750061ec440e69712be3a830cc202, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:37.880477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:37.880477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:10:37.880477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:10:37.882433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf937a74-09be-4a5c-9c0d-8ca9d62bb160 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:37.906427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 748/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:10:37 2022] PUT /identity/v3/projects/dd8cdd492b3e4529a2e972e6ebff9ade/users/0d4750061ec440e69712be3a830cc202/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:10:37.925009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944701-c961-4c66-86a5-91151824f985 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:37.925379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944701-c961-4c66-86a5-91151824f985 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:37.925650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3944701-c961-4c66-86a5-91151824f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:37.982147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c3944701-c961-4c66-86a5-91151824f985 None None] MFA Rules not processed for user `0d4750061ec440e69712be3a830cc202`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:10:37.983358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c3944701-c961-4c66-86a5-91151824f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:38.059130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 754/1502] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:10:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:10:38.072453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.072771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.073045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.074407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 749/1503] 10.222.0.46 () {54 vars in 964 bytes} [Thu Sep 1 07:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:10:38.079108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929921a-0103-4c7e-b7ff-9841dd0e5ba0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.079393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929921a-0103-4c7e-b7ff-9841dd0e5ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.079664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929921a-0103-4c7e-b7ff-9841dd0e5ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.111374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:38.155778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6929921a-0103-4c7e-b7ff-9841dd0e5ba0 None None] MFA Rules not processed for user `92e78fff3061449ba36aafbc1b3f7eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:10:38.156965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6929921a-0103-4c7e-b7ff-9841dd0e5ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:38.231003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 755/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Sep 1 07:10:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:10:38.240504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:38.242042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:38.292231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:38.345939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65e03-db6d-43fd-ad31-95c6cfce81c7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:38.418364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:38.418885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.419158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.419437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.420981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:38.515667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:38.517079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:38.520997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:38.536503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 750/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:10:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:10:38.581204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38af8d-a9f9-410c-9084-5960bfe2c4f5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.581639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38af8d-a9f9-410c-9084-5960bfe2c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.581771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38af8d-a9f9-410c-9084-5960bfe2c4f5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.583045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 756/1506] 10.222.0.46 () {52 vars in 830 bytes} [Thu Sep 1 07:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:10:38.586541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:38.587727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:38.688450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2b9a5-93c7-4bb1-9361-ad9a1ef8d900 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:38.781229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:38.781880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.782153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.782422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] PATH_INFO: `/v3/users/0d4750061ec440e69712be3a830cc202` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.794476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4750061ec440e69712be3a830cc202)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:38.795521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:38.795521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:10:38.795521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:10:38.796716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:38.806070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:38.860247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4750061ec440e69712be3a830cc202'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:38.874138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4750061ec440e69712be3a830cc202'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:38.883704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4750061ec440e69712be3a830cc202'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:38.884166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4750061ec440e69712be3a830cc202'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:38.905992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4750061ec440e69712be3a830cc202'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:38.911533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 751/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:10:38 2022] DELETE /identity/v3/users/0d4750061ec440e69712be3a830cc202 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:10:38.915953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:38.916243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:38.916546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:38.972944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:10:38.974104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:39.047215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:39.061393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 757/1508] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:10:39.073124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:39.074392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:39.172691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21c205-7363-4192-9f69-600104403c58 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:39.228522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:39.249995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:39.250606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:39.250883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:39.251152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:39.253067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:39.358126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:39.359569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496ec8c-4352-4b72-9988-eb7e94d305bf service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:39.378376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 752/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:10:39.697036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:10:39.698392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:10:39.710563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16da462-9e3c-4371-9968-e196b514ac21 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:10:39.812477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:10:39.813104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:10:39.813410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:10:39.813680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] PATH_INFO: `/v3/projects/dd8cdd492b3e4529a2e972e6ebff9ade` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:10:39.823580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8cdd492b3e4529a2e972e6ebff9ade)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:10:39.824595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:10:39.824595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:10:39.824595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:10:39.825822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:10:39.883386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8cdd492b3e4529a2e972e6ebff9ade', 'request_id': 'req-099e8f51-0b69-4683-a44b-a25973e00ee2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:39.892915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8cdd492b3e4529a2e972e6ebff9ade', 'request_id': 'req-099e8f51-0b69-4683-a44b-a25973e00ee2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:39.901040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8cdd492b3e4529a2e972e6ebff9ade', 'request_id': 'req-099e8f51-0b69-4683-a44b-a25973e00ee2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:39.934239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] The token cache is being invalidate because project dd8cdd492b3e4529a2e972e6ebff9ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:10:39.934527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:39.934965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-099e8f51-0b69-4683-a44b-a25973e00ee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:10:39.936385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 758/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:10:39 2022] DELETE /identity/v3/projects/dd8cdd492b3e4529a2e972e6ebff9ade => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:10:49.375849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=62 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:10:49.931372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:12:07.052053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:07.052652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:07.052652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:07.054023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:07.058984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7562f9-2cc3-4efe-8e27-0af15dca5cea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:07.059419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7562f9-2cc3-4efe-8e27-0af15dca5cea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:07.059524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7562f9-2cc3-4efe-8e27-0af15dca5cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:07.141589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc7562f9-2cc3-4efe-8e27-0af15dca5cea None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:07.143670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc7562f9-2cc3-4efe-8e27-0af15dca5cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:07.245937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:07.399598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:07.401787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:07.523553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e0410-e4de-4cee-bb58-2128240c87e9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:07.529009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:07.529646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:07.529927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:07.530206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:07.539108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:07.540729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28ffc1-4a35-4cd1-a1a2-66b6a700cf7a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:07.550114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 754/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:08.370993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7c42-0106-44f9-bd3d-e046ab9eaf06 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:08.371555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7c42-0106-44f9-bd3d-e046ab9eaf06 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:08.371555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d7c42-0106-44f9-bd3d-e046ab9eaf06 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:08.372865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:08.377602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03769482-62fe-4c3c-bee2-a706b6d9a7bc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:08.378013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03769482-62fe-4c3c-bee2-a706b6d9a7bc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:08.378155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03769482-62fe-4c3c-bee2-a706b6d9a7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:08.398127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-03769482-62fe-4c3c-bee2-a706b6d9a7bc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:08.399309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-03769482-62fe-4c3c-bee2-a706b6d9a7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:08.406971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:08.560808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7c42-0106-44f9-bd3d-e046ab9eaf06 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:08.575064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d7c42-0106-44f9-bd3d-e046ab9eaf06 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:08.580700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:08.581336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:08.581618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:08.581890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:08.589998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:08.591568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d8d62-9e29-4b94-83b8-a4b5ee34a540 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:08.601257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 761/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:08.902115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb95d-989b-46a2-b553-9af498ed14fa None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:08.902306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb95d-989b-46a2-b553-9af498ed14fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:08.902582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb95d-989b-46a2-b553-9af498ed14fa None None] PATH_INFO: `/v3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:08.903923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 756/1517] 10.222.0.25 () {54 vars in 925 bytes} [Thu Sep 1 07:12:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:12:08.910233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde1241-3fb4-43ca-b027-0d511f027d6b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:08.911142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde1241-3fb4-43ca-b027-0d511f027d6b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:08.911423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde1241-3fb4-43ca-b027-0d511f027d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:08.969575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5bde1241-3fb4-43ca-b027-0d511f027d6b None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:08.970769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bde1241-3fb4-43ca-b027-0d511f027d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:08.987642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:12:09.041970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 762/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 07:12:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:18.009711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101b93c-d9eb-4d06-abac-36b10289a112 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:18.010241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101b93c-d9eb-4d06-abac-36b10289a112 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:18.010241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101b93c-d9eb-4d06-abac-36b10289a112 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:18.011495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 757/1519] 10.222.0.46 () {54 vars in 961 bytes} [Thu Sep 1 07:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:18.016315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856342-1d08-4df0-8bfb-ea898aad8d07 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:18.016704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856342-1d08-4df0-8bfb-ea898aad8d07 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:18.016860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e856342-1d08-4df0-8bfb-ea898aad8d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:18.037162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8e856342-1d08-4df0-8bfb-ea898aad8d07 None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:18.038362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e856342-1d08-4df0-8bfb-ea898aad8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:18.046812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 763/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Sep 1 07:12:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:18.054097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101b93c-d9eb-4d06-abac-36b10289a112 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:18.067283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101b93c-d9eb-4d06-abac-36b10289a112 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:18.072192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:18.072769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:18.073044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:18.073342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:18.081315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:18.083433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17989174-5509-4396-9f87-01935623d8e6 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:18.092707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 758/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:19.822637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab36ee-f832-49a2-b77b-7f4c085c3b4b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:19.823149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab36ee-f832-49a2-b77b-7f4c085c3b4b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:19.823291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab36ee-f832-49a2-b77b-7f4c085c3b4b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:19.824509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:19.829234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34554-cff4-40ca-bcdd-8aa6ba1450b6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:19.829650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34554-cff4-40ca-bcdd-8aa6ba1450b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:19.829810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34554-cff4-40ca-bcdd-8aa6ba1450b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:19.849396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-efd34554-cff4-40ca-bcdd-8aa6ba1450b6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:19.850583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efd34554-cff4-40ca-bcdd-8aa6ba1450b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:19.858310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:20.014130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab36ee-f832-49a2-b77b-7f4c085c3b4b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:20.028432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab36ee-f832-49a2-b77b-7f4c085c3b4b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:20.033831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:20.034419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:20.034698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:20.034971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:20.042820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:20.044390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a48f31-237d-4872-ad89-00bf1cac0db6 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:20.054037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 765/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:20.866795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff21-6808-43d3-a782-62cdc81339f6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:20.867331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff21-6808-43d3-a782-62cdc81339f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:20.867331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bff21-6808-43d3-a782-62cdc81339f6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:20.868670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 07:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:20.873387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff3e7-2df2-460f-b668-5c0a05f12003 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:20.873790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff3e7-2df2-460f-b668-5c0a05f12003 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:20.873932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ff3e7-2df2-460f-b668-5c0a05f12003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:20.894515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-533ff3e7-2df2-460f-b668-5c0a05f12003 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:20.896180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-533ff3e7-2df2-460f-b668-5c0a05f12003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:20.903781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 07:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:21.059425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bff21-6808-43d3-a782-62cdc81339f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:21.073469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bff21-6808-43d3-a782-62cdc81339f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:21.078890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:21.079471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:21.079745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:21.080012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:21.088422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:21.090031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55ca6-e369-45c9-8ca0-614999830856 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:21.099636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 761/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:24.769033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da408a94-68da-41b6-a421-855b89530705 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:24.770424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da408a94-68da-41b6-a421-855b89530705 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:24.770424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da408a94-68da-41b6-a421-855b89530705 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:24.771478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:12:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:24.776541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31d3d-e322-4225-9997-eddb1e2d2889 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:24.776953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31d3d-e322-4225-9997-eddb1e2d2889 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:24.777082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf31d3d-e322-4225-9997-eddb1e2d2889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:24.796646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8cf31d3d-e322-4225-9997-eddb1e2d2889 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:24.797882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8cf31d3d-e322-4225-9997-eddb1e2d2889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:24.805688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:25.092825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da408a94-68da-41b6-a421-855b89530705 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:25.094540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da408a94-68da-41b6-a421-855b89530705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:25.170725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da408a94-68da-41b6-a421-855b89530705 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:25.174272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:25.174871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:25.175156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:25.175437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:25.184124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:25.185597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01769c2-5528-452c-8f9f-098278bf7e30 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:25.195311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 768/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:27.073910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:27.074438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:27.074438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:27.075783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:27.080858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1078-0dbe-480e-a0ca-66effdf42c58 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:27.081325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1078-0dbe-480e-a0ca-66effdf42c58 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:27.081434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb1078-0dbe-480e-a0ca-66effdf42c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:27.100529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-04bb1078-0dbe-480e-a0ca-66effdf42c58 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:27.101748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-04bb1078-0dbe-480e-a0ca-66effdf42c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:27.109448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:27.398266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:27.399987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:27.413343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a490-aab8-434b-b3db-61dd30fc17b5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:27.416792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:27.417479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:27.417756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:27.418027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:27.427340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:27.428759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184258b-e255-4938-8f6b-c37e00fd080e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:27.437998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 764/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:30.825662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:30.826178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:30.826276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:30.827568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:12:30.832804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e22c8-a4bf-4bae-9111-93a440d4c28e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:30.833215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e22c8-a4bf-4bae-9111-93a440d4c28e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:30.833388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e22c8-a4bf-4bae-9111-93a440d4c28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:30.840949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:12:30.853594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d07e22c8-a4bf-4bae-9111-93a440d4c28e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:30.854796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d07e22c8-a4bf-4bae-9111-93a440d4c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:30.862631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:31.414345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:31.416030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:31.479620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:12:31.492796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83447fe-b372-4dca-a8dd-1dfdf297a64f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:31.496385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:31.496977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:31.497288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:31.497562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:31.506529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:31.507938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93a39d-973b-4e9f-8c5f-f5f867cd8c40 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:31.518209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 771/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:33.424679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:33.425295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:33.425295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:33.426619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 07:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:12:33.431671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e30a-bc49-4ba8-ab9c-cda464cd83db None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:33.432074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e30a-bc49-4ba8-ab9c-cda464cd83db None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:33.432217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2e30a-bc49-4ba8-ab9c-cda464cd83db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:33.451646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fcd2e30a-bc49-4ba8-ab9c-cda464cd83db None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:12:33.452824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fcd2e30a-bc49-4ba8-ab9c-cda464cd83db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:33.461056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 07:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:12:34.010501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:12:34.012180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:12:34.025431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff7f7c-346a-48fe-9cd0-53fb39065ab7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:12:34.028846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:12:34.029456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:12:34.029727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:12:34.029993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:12:34.038673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:12:34.040079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc4d79-d3e2-49d5-82d7-38127a8decb3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:12:34.049190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 767/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:12:43.446969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=11 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:12:44.045160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:57.840286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:57.842017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:57.842017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:57.869517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:57.870714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:57.882291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 773/1540] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:57.891296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:57.891912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:57.891912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:57.926037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:57.927500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:57.939614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 768/1541] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:57.944821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:57.960831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d506bc-2db5-4082-aa0c-88c499e2ac18 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:57.970044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:57.970669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:57.970944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:57.971214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:57.971863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:57.972160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:57.974434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90327a95-0dda-41e2-9c3f-094d761a14d0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:57.999815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 774/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:58.014460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.024994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cb22e-e529-437e-99b6-cde579402bbc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.026295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e753-e0fe-4352-9257-9585442b4e3a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.026890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e753-e0fe-4352-9257-9585442b4e3a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.027174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518e753-e0fe-4352-9257-9585442b4e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.055511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.056216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.056493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.056767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.057469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.059377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.084398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a518e753-e0fe-4352-9257-9585442b4e3a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.085613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a518e753-e0fe-4352-9257-9585442b4e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.135024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 775/1543] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.144546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a830e-b600-4156-a89e-7439bb5cc157 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.144942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a830e-b600-4156-a89e-7439bb5cc157 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.145146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a830e-b600-4156-a89e-7439bb5cc157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.183295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 769/1544] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:58.188103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8f6a830e-b600-4156-a89e-7439bb5cc157 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.189174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.193155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8f6a830e-b600-4156-a89e-7439bb5cc157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.439830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6de9fa-e2e8-44ea-8dc2-96adad7c77e0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.441083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 776/1545] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.443520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.444092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.444604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.467653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.468293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.468571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.468840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.469508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.472864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:13:58.472864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:13:58.472864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:13:58.472864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010beb1-3b79-4347-bd1d-1095c8865969 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.490827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.492034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.510916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 777/1546] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.514346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.535798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 770/1547] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/users => generated 296 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:58.542514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e722d-0431-4eeb-82a9-940e2a70ddf5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.542838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e722d-0431-4eeb-82a9-940e2a70ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.543080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e722d-0431-4eeb-82a9-940e2a70ddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.547626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b7748-5a16-4c5a-be3a-54f34bc085d3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.569998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.571183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.571712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.572250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.573201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:58.573729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.575819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15632ec2-8b6f-421f-a387-e0b6a52355d4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.589194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-207e722d-0431-4eeb-82a9-940e2a70ddf5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.590563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 778/1548] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:58.590690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-207e722d-0431-4eeb-82a9-940e2a70ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.591644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.591933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.592203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.602193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 771/1549] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.603276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae878d-c3b6-41d6-b407-aca0bb06fb82 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.603554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae878d-c3b6-41d6-b407-aca0bb06fb82 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.603828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae878d-c3b6-41d6-b407-aca0bb06fb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.618041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.619687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.623132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e6ae878d-c3b6-41d6-b407-aca0bb06fb82 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.624284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e6ae878d-c3b6-41d6-b407-aca0bb06fb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.637977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 779/1550] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.637977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.640672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 772/1551] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.641950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514432f6-5ecd-44ef-9d1a-a4a66049783d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.642227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514432f6-5ecd-44ef-9d1a-a4a66049783d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.642503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514432f6-5ecd-44ef-9d1a-a4a66049783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.648669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaabbdf-3d08-4d97-9283-f6fec1cfe0cd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.654235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.654938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.655218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.655486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.657298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.658956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed75c9-b57d-4484-8ba3-0f1d68204c2a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.664675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-514432f6-5ecd-44ef-9d1a-a4a66049783d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.665943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-514432f6-5ecd-44ef-9d1a-a4a66049783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.672016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 780/1552] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:58.672016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.672016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.672016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.677481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 773/1553] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.678368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fe03-4944-4056-b0e9-69577c3e2f23 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.678651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fe03-4944-4056-b0e9-69577c3e2f23 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.678930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fe03-4944-4056-b0e9-69577c3e2f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.693050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.694293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.698460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2693fe03-4944-4056-b0e9-69577c3e2f23 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.699617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2693fe03-4944-4056-b0e9-69577c3e2f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.702169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 781/1554] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.703049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.709990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 774/1555] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.711284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e8e3e-dc80-4698-94f7-b4e9b2298bb0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.711638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e8e3e-dc80-4698-94f7-b4e9b2298bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.711976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e8e3e-dc80-4698-94f7-b4e9b2298bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.713315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c070e25-6f5e-45a4-9e77-be85534db6c1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.718505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.719101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.719382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.719846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.721269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.722302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:13:58.722302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:13:58.722302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:13:58.723526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.732500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-842e8e3e-dc80-4698-94f7-b4e9b2298bb0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.733721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-842e8e3e-dc80-4698-94f7-b4e9b2298bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.743686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 775/1556] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.745341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.745782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.746062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.752475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 782/1557] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:58.753724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.769465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.770409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.821372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea7ed4-bcce-4a16-9cb4-829de3b55631 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.826130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.826724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.827002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.827272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.827931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:58.828236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.828869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 776/1558] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.829737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.829865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3000869-00e5-4584-a41f-71360c9df17a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.841457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d3f55-eb68-4931-991c-8b0e607b4fe0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.842823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 783/1559] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:58.844018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f267ed-dd1f-4b5b-87e9-77a4820885d1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.844673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f267ed-dd1f-4b5b-87e9-77a4820885d1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.844952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f267ed-dd1f-4b5b-87e9-77a4820885d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.847944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.848644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.849024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.849478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.850223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.851830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486eea1c-023c-4f5c-80d1-779d1c1f6899 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.852786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:58.867136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 777/1560] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:58.867136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.867136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.867136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.873589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d3f267ed-dd1f-4b5b-87e9-77a4820885d1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.874794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3f267ed-dd1f-4b5b-87e9-77a4820885d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.883827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 784/1561] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.884993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.885313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.885584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.889033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.890286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.898169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 778/1562] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.898948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.904266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:58.905446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:58.906058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:58.912027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb795851-ef5a-48b9-947c-57904d74ec7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.914488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 785/1563] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:58.914488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.918296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.918296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.918296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.918761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.919491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:58.919900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.921002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:13:58.921002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:13:58.921002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:13:58.922151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.924635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efddba0b-39a6-49b2-9b5a-37ea0501f532 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.929693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.930380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.930662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.930934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.931557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.933476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.934194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 779/1564] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:58.934979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.948011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf92a-877f-4045-b044-a3c613b389b4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.953853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.954438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.954741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.955010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.955643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:58.955941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.956700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 786/1565] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:58.957449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.957582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.974828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 780/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:58.974998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90d7c-785a-4ef5-b900-931c4922e52c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.975089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:58.978681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.979331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.979614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.979904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:58.980529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:58.982462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:58.984577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0010d86-b701-4936-8474-2acab8580410 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:58.989163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:58.990307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:58.990584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:58.990856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] PATH_INFO: `/v3/projects/c1705474b7954cf1926e9beb2d548587/users/db4f0928502a43b490de01afc587a846/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.005907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1705474b7954cf1926e9beb2d548587, user_id=db4f0928502a43b490de01afc587a846, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.008549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.010804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 787/1567] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:58 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.012110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.021871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 781/1568] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:13:58 2022] PUT /identity/v3/projects/c1705474b7954cf1926e9beb2d548587/users/db4f0928502a43b490de01afc587a846/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:13:59.025980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.073832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea38a-fee6-4514-a21a-f247efd5e682 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.086607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c22790-c62e-4da1-a395-f87f4c2e45de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.091241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.091859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.092137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.092409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.093149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:59.093469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.095043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.106708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 782/1569] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:59.107508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.119258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b2a9-b0ba-488b-98b3-47eb0f3f4c31 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.123805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.124387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.124662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.124933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.125594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:59.125893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.127461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.132550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.133251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.133534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.133809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.134449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:13:59.134752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.136846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.140459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 783/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:59.141025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.152143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18bcc3-eaa0-4fdf-ba9c-ab1ff93d94de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.152873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 788/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:13:59.153595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.156911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.157520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.157852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.158128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.158768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.160636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.165280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b7fca-d4f4-4e6f-93fb-23078995d3e5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.170794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.171406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.171687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.171988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.172631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.174284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.186561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 789/1572] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.187336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.194152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 784/1573] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.194152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.199637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd882c-a475-4e25-a9b0-c146b9d44847 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.238593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:59.253428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4dcfc-d9d5-4727-8233-fd759a963e24 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.258341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.258936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.259142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.259327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.259465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.259872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.260088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.260212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.260438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.261036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.262034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11fee4-dbb8-4dc9-8936-af216be20835 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.262948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.279279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:59.285197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 785/1574] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.287308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.287962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.288424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.293481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 790/1575] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.294446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.307919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:59.309139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:59.359534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18ede8-40d1-46aa-87dc-11a29f9ee300 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.364427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.365028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.365348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.365743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.366366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.368276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.368987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 786/1576] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:13:59.369854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.379963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322d1d-3308-4ee4-bd0c-dcf22caba1f3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.385332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.386059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.386334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.386609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.387233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.389180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.396741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 791/1577] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.397587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.420772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 787/1578] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.421706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.464821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468bf64-fdb1-48ba-96bb-14ae44800a7f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.481799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fbfd0-25e7-46a0-9faf-990016b420b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.486421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.487020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.487302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.487574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.488208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.489870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.499915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 788/1579] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.500759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.510642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8aca-4420-40fa-8955-90aaee401286 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.515604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.516320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.516595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.516866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.517515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.518050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:59.519416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.531110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.532545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.532999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.533383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.534212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.537283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.542975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 789/1580] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.543691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.553196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d0f8-85cf-4de2-8491-1d6a82965c19 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.557828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.558408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.558679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.558951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.559574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.561181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.565150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 792/1581] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.566016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.574534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 790/1582] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.574534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.621327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:59.626254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7176f-5601-43e5-90f2-4446e51af034 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.629140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf7ec9-411e-4628-b9ae-48d28e359152 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.634760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.635360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.635638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.635903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.636559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.638512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.659916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 793/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.660697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.669771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3968794-b342-4bef-b2cd-1ed3dc29baa2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.674278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.674847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.675122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.675391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.675992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.678516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.680355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.680959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.681266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.681540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.690468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.692121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.700485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 794/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.703015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 791/1585] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:13:59.704504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.704656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.714416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ff024-a4c2-4312-b0b7-8a31edef327b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.714604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734366f-c24f-4ef1-bb42-631bbce5040a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.719277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.719443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.719873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.720489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.720617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.720737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.720839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.721029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.721260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] PATH_INFO: `/v3/projects/5be5442e873f45e5a98a95b58a9a7ede/users/9d28e747db8044e18ba2ce4a100cbaaa/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.722964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.738557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5be5442e873f45e5a98a95b58a9a7ede, user_id=9d28e747db8044e18ba2ce4a100cbaaa, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.741987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.753199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 795/1586] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.753199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.760713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 792/1587] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:13:59 2022] PUT /identity/v3/projects/5be5442e873f45e5a98a95b58a9a7ede/users/9d28e747db8044e18ba2ce4a100cbaaa/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:13:59.765210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbd814-0f7c-47f6-a4ec-5ead921ef7a1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.766197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.841027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254981e-f7be-4bd9-8b10-f573d64fa80e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.846222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.846917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.847264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.847534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.848182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.848303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.849535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.850041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.850413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.850664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.851243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.853174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.865292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 796/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.868665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.876223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 793/1589] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:13:59.877054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.880555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fcae9-c5e3-4552-a5ba-4c16750043cd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.886232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.887529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.887914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.888291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.888566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57d52a-9dd7-4413-a48c-a436d9bafe2e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.889170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.890891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.895048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.895695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.895973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.896244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.896910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.899222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.902355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 797/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.906251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.913571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 794/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.913571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.921891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1028c8-4071-4917-991f-ac4fa97537e4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.928943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.929817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.930243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.930829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.930982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138e070-4502-4e4e-9daf-6aa92ac3a742 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.931986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.934266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6c5bf-9927-4d89-935c-6907533ada0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.938938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.939891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.939891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.940140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.941320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.943119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.947898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 795/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.949245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.949591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.950542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.957164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 798/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:13:59.958506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:13:59.971664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57456e54-76c7-4993-a22d-c56471d0cc40 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:13:59.977553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:13:59.978037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:13:59.978795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:13:59.978795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:13:59.979490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:13:59.981177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:13:59.985693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] MFA Rules not processed for user `9d28e747db8044e18ba2ce4a100cbaaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:13:59.987163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:13:59.987915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:13:59.997913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 799/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.000196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.011120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21affd2-7dae-49dc-b500-b581d01a1797 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.015787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.016562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.016758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.017104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.017910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.019541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.030896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 800/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.031346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.041755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe811a-31c5-4783-92d5-ba1d2f8ab1d7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.044099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:00.048452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.049698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.049698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.049927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] PATH_INFO: `/v3/projects/9f10e8c5e7694446a81ea085a367342f/users/b461f8c619694b89ad36f3b71b5b434e/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.068473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f10e8c5e7694446a81ea085a367342f, user_id=b461f8c619694b89ad36f3b71b5b434e, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.069570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:14:00.069570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:14:00.069570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:14:00.071779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.080120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 796/1596] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:13:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:00.081263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.094002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2442e4-13a9-4308-ab22-d31a2a01a7ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.094206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 801/1597] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/9f10e8c5e7694446a81ea085a367342f/users/b461f8c619694b89ad36f3b71b5b434e/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.095206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.163308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.164121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.164121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.164384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.166337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.168303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.169785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e19fe-f799-41bf-a542-f952184a2a35 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.181312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.182059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.182403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.182867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.183771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.184366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 797/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.185838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.189768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.200458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3a17-50b1-4805-9648-70caf222090b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.202741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 802/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.203725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.210758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.211485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.211771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.212048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.212861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.215049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.217856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612b7d4-84ef-4df9-b152-4bcc237e1e9a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.229098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 798/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.229999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.237419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.239934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.240776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.240776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] PATH_INFO: `/v3/projects/c4f0d85a8fee4b2a95d2c05d30afc078/users/c729490faaa643ffbbdb6e5178b28dd2/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.244623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509886ea-1689-4a63-813a-31964e65dca2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.249361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.249964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.250248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.250524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.251161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.252779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.257746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f0d85a8fee4b2a95d2c05d30afc078, user_id=c729490faaa643ffbbdb6e5178b28dd2, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.260990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c7158-d8ca-4c80-888b-c9bce8197bae admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.268091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 799/1601] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:00.269162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.280058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89294-e711-4a49-b0e3-97680d5bcc07 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.285522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 803/1602] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/c4f0d85a8fee4b2a95d2c05d30afc078/users/c729490faaa643ffbbdb6e5178b28dd2/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.287879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.288190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.288626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.322257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] MFA Rules not processed for user `b461f8c619694b89ad36f3b71b5b434e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:00.323612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:00.348500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.350125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.350417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.350694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.351336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.354019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.393482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 800/1603] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:00.393482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.418049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 804/1604] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:00.422269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.433098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c9129-96b9-402a-ad6d-128ee10da3d2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.445250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:00.461777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:00.475465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391044ac-f3a5-42ad-be94-eec1021be535 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.485442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.486859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.487417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.488338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] PATH_INFO: `/v3/projects/4e94525a3f1c4bc5a6a8846e3a1d134e/users/2a306c3e658e4a52a8618cf48117953b/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.509703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e94525a3f1c4bc5a6a8846e3a1d134e, user_id=2a306c3e658e4a52a8618cf48117953b, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.510843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.511725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.512701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.512924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.513591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] PATH_INFO: `/v3/projects/917622d952994fae9715d8c50809324a/users/5af41c1833234fa1985029653ef6d320/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.530840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 801/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/4e94525a3f1c4bc5a6a8846e3a1d134e/users/2a306c3e658e4a52a8618cf48117953b/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.532077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.543884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917622d952994fae9715d8c50809324a, user_id=5af41c1833234fa1985029653ef6d320, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.549049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.566258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 805/1606] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/917622d952994fae9715d8c50809324a/users/5af41c1833234fa1985029653ef6d320/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.567961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.633082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d3e4-91b0-4f4a-ab61-02204c82ffb2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.635009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91939028-6429-43bd-8025-e11624e5ba30 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.638118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.638898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.639187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.639459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] PATH_INFO: `/v3/projects/78f0be015ba44cdfa6c64038e16bd5cb/users/561fa6572e0b4b949d1c09ee9e9d5e66/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.640208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.640961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.641372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.641737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] PATH_INFO: `/v3/projects/e2774daeaf2945eea5b6f33ad4ea2bdf/users/2096dd4d4b6f49fc8572a4bcd9689248/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.658846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2774daeaf2945eea5b6f33ad4ea2bdf, user_id=2096dd4d4b6f49fc8572a4bcd9689248, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.660046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f0be015ba44cdfa6c64038e16bd5cb, user_id=561fa6572e0b4b949d1c09ee9e9d5e66, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.661865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d05b2-3c87-437f-80e1-148fa4fe1998 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.664806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.682815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 806/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/e2774daeaf2945eea5b6f33ad4ea2bdf/users/2096dd4d4b6f49fc8572a4bcd9689248/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.684415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035ce60-041e-4021-8512-985e4ab0d715 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.685248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035ce60-041e-4021-8512-985e4ab0d715 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.685678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035ce60-041e-4021-8512-985e4ab0d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.688304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 802/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:00 2022] PUT /identity/v3/projects/78f0be015ba44cdfa6c64038e16bd5cb/users/561fa6572e0b4b949d1c09ee9e9d5e66/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:00.689319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.719848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1035ce60-041e-4021-8512-985e4ab0d715 None None] MFA Rules not processed for user `c729490faaa643ffbbdb6e5178b28dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:00.721706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1035ce60-041e-4021-8512-985e4ab0d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:00.788558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c4066-d972-4d23-8b58-5547dab75757 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.796387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.797886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.798234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.798594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.800133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.802477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.835876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 803/1609] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:00.836108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 807/1610] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:00.836883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:00.837239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f7da-04fe-458e-bce9-905661819840 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.837532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f7da-04fe-458e-bce9-905661819840 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.837812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6f7da-04fe-458e-bce9-905661819840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.856887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da10237-f7a8-40f9-8a04-b513a3bb7643 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:00.862099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:00.862814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.863238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.863664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.864983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:00.867164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f6180-9c8f-4726-ae63-1e82f6b7bb54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:00.869199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f8c6f7da-04fe-458e-bce9-905661819840 None None] MFA Rules not processed for user `2a306c3e658e4a52a8618cf48117953b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:00.871003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f8c6f7da-04fe-458e-bce9-905661819840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:00.884631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:00.905629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 804/1611] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:00.910462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226a94-e8fa-4381-8d4b-66fad8cfb17a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.910751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226a94-e8fa-4381-8d4b-66fad8cfb17a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.911557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b226a94-e8fa-4381-8d4b-66fad8cfb17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.940994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:00.953316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3b226a94-e8fa-4381-8d4b-66fad8cfb17a None None] MFA Rules not processed for user `5af41c1833234fa1985029653ef6d320`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:00.954686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3b226a94-e8fa-4381-8d4b-66fad8cfb17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:00.965684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 808/1612] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:00.967927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:00.968708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:00.968993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:00.992859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:00.994283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:01.069016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 805/1613] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:01.070486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.070946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.071221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.074874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 809/1614] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:01.076136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.101342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:01.102566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:01.107484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fadb5-fd94-4c59-a3e9-03bd23d61b89 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.113019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.113772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.114671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.118072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.118213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 806/1615] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:01.118324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.118417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.118504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.129031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 810/1616] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:01.129372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.141675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57508a81-008a-44f7-928d-1ede697a31a4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.152862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.154288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.154880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.155782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.156622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.158835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45316d-217d-43ab-bc8c-e654a8525e1a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.195496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 811/1617] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:01 2022] POST /identity/v3/users => generated 294 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:01.200194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f906-8adb-41e7-b635-dd49fa7d2f06 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.201148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f906-8adb-41e7-b635-dd49fa7d2f06 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.201615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146f906-8adb-41e7-b635-dd49fa7d2f06 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.204048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 812/1618] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 07:14:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:01.205605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f906-8adb-41e7-b635-dd49fa7d2f06 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.211268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f5012-6fba-4d54-be3c-542d9b727f13 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.216217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.216871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.217244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.217985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.230596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146f906-8adb-41e7-b635-dd49fa7d2f06 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.236689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.237338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.238045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.238318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.238958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.241614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.293186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 813/1619] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:01 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:01.293186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.311868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.313495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.324480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 807/1620] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:01.326414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.373880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d23f0d-29a3-43df-bb5b-616f8b7247c5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.378826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.380160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.381235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.381699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.393854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:01.400889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:01.401415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ecba9-2e77-4e1f-95bc-060bfe81c89b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.409914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.410592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.411408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.412764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.541712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.541712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.542283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.542283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.542597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 814/1621] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:01.542690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 808/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:01.542690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.556766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.559891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75419c8b-e825-465c-a2d2-b1211dd7532a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.566554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.567313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.567597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.567885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.568529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.570354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39355b-c74a-411f-8e84-c0117bb40a17 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.592049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef295c-883e-41bd-90c5-186bbf3a2a93 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.599510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 809/1623] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:01.601667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.601941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.602411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.609812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.610848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.611201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.611599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] PATH_INFO: `/v3/projects/1c1ce987708144a69d878f1fd401a7d5/users/5d7d310d4afa49f183e48aba44b2ac0f/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.646820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1ce987708144a69d878f1fd401a7d5, user_id=5d7d310d4afa49f183e48aba44b2ac0f, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.650214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.675224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 815/1624] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:01 2022] PUT /identity/v3/projects/1c1ce987708144a69d878f1fd401a7d5/users/5d7d310d4afa49f183e48aba44b2ac0f/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:01.676252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.714742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:01.716411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:01.827170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c2a2c-2aff-4fd1-8e64-97ac212749d6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.829543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 810/1625] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 07:14:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:01.830537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.846031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.846773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.847058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.848268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.849742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.855342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.857699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43db7a7-1e53-4ca4-a59a-a7d071787be9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.872648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:01.874552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:01.874881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:01.875963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] PATH_INFO: `/v3/projects/795843e364604efd8d73875152ea8ffc/users/e81f3a96b1c84c1b8428888d205ca0a6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:01.907663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 816/1626] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:01 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:01.913394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:01.919521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795843e364604efd8d73875152ea8ffc, user_id=e81f3a96b1c84c1b8428888d205ca0a6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:01.924194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:01.940948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9003be-072c-40b1-b327-a7272d8e12c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:01.949818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 811/1627] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:01 2022] PUT /identity/v3/projects/795843e364604efd8d73875152ea8ffc/users/e81f3a96b1c84c1b8428888d205ca0a6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:01.951357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.049737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:02.066529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.067800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.071230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.071607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.072465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.077083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.078585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41b51-76a0-49b1-ab88-ba475d2cd131 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.087955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.088784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.089201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.089674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.108199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 817/1628] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:02.111324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.143849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a25c49-59b3-4b76-8ec5-be28430d5d8b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.154974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.155790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.156197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.156483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.157181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.158881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.213015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 818/1629] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:02.213015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.232124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:02.233079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.239849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.257891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd89eb-0848-42fd-9e6a-b6f314731dc7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.262822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.263441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.263719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.263985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.264629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.266325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.267234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 812/1630] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:02.268744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.316796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 819/1631] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:02.316796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.321267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793265e-a9d8-408b-ba32-4c36c84fe413 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.342673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.346027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.350787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.351494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13329039-71c4-446d-9c0c-f4771c69b21b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.352921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] PATH_INFO: `/v3/projects/1c1ce987708144a69d878f1fd401a7d5/users/5d7d310d4afa49f183e48aba44b2ac0f/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.362360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c1ce987708144a69d878f1fd401a7d5, user_id=5d7d310d4afa49f183e48aba44b2ac0f, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.365458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.365684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.365948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.366216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.366859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.368456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.387364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.400754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 820/1632] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:02.401709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.424071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a020d1-cfd8-42e3-bca7-3cdb5711a956 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.432552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 813/1633] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:02 2022] PUT /identity/v3/projects/1c1ce987708144a69d878f1fd401a7d5/users/5d7d310d4afa49f183e48aba44b2ac0f/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:02.433658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.606244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.607150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.607535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.613909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd179b-1ec5-464d-b77f-4384d45a95b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.619032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] PATH_INFO: `/v3/projects/795843e364604efd8d73875152ea8ffc/users/e81f3a96b1c84c1b8428888d205ca0a6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.623515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795843e364604efd8d73875152ea8ffc, user_id=e81f3a96b1c84c1b8428888d205ca0a6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.628061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b1aeb-51ee-4487-a57c-3bd85a91aaf5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.633249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.638264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.640070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.640357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] PATH_INFO: `/v3/projects/09a310c9dba84fce87f63338e2ada969/users/a1303a49a7bf48c0b110e10f20bc59e1/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.654693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 821/1634] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:02 2022] PUT /identity/v3/projects/795843e364604efd8d73875152ea8ffc/users/e81f3a96b1c84c1b8428888d205ca0a6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:02.656420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b139f-ec87-44b9-9bfe-2ad62bc8641f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.658377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b139f-ec87-44b9-9bfe-2ad62bc8641f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.660989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b139f-ec87-44b9-9bfe-2ad62bc8641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.675172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a310c9dba84fce87f63338e2ada969, user_id=a1303a49a7bf48c0b110e10f20bc59e1, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:02.678987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d95c5-351c-46c8-ab04-3f17dbcd9fc6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:02.701852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 814/1635] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:02 2022] PUT /identity/v3/projects/09a310c9dba84fce87f63338e2ada969/users/a1303a49a7bf48c0b110e10f20bc59e1/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:02.703631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9965638-5188-4005-b221-cf80f854f550 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.704140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9965638-5188-4005-b221-cf80f854f550 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.704694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9965638-5188-4005-b221-cf80f854f550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.750833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c59b139f-ec87-44b9-9bfe-2ad62bc8641f None None] MFA Rules not processed for user `5d7d310d4afa49f183e48aba44b2ac0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:02.751188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d9965638-5188-4005-b221-cf80f854f550 None None] MFA Rules not processed for user `e81f3a96b1c84c1b8428888d205ca0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:02.752745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c59b139f-ec87-44b9-9bfe-2ad62bc8641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:02.757281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9965638-5188-4005-b221-cf80f854f550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:02.808534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:02.851992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 815/1636] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:02.853342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.853630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.854034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.887690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 822/1637] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:02.893600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870d6f-ac58-4a04-97d3-08f04b09e64b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.893885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870d6f-ac58-4a04-97d3-08f04b09e64b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.894160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c870d6f-ac58-4a04-97d3-08f04b09e64b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.900619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 823/1638] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 07:14:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:02.901537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870d6f-ac58-4a04-97d3-08f04b09e64b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:02.921309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] MFA Rules not processed for user `a1303a49a7bf48c0b110e10f20bc59e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:02.922897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:02.967816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c870d6f-ac58-4a04-97d3-08f04b09e64b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:02.972375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:02.973042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:02.973374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:02.974009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:02.990285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:03.009155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 816/1639] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:03.010011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.022162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9e690-ac51-4318-be78-730c858bbd57 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.026462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.027251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.027897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.028195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.040613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.042972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af80c7e-af2a-4257-824e-ea47483ae549 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.054397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 817/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:03.055538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.055828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.056112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.080002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:03.080531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:03.086560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.088008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.109020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 824/1641] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:03.110419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.247713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 818/1642] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:14:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:03.262258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66cd5c-3bcf-4752-8a29-e7804f01a6a6 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.267329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.267977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.268256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.268525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.269325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.271655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.309632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.323945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 825/1643] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:03 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:03.324184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb8a28-6b91-41fb-ab13-48d92df07510 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.327290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.449870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:03.470431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.471215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.471498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.471771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.522689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cd2c-89f2-4a9f-bdae-bf83c7dd7f27 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.530449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.531192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.531483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.531750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.532400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.534989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.633199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 826/1644] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:03 2022] POST /identity/v3/users => generated 299 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:03.638085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.684241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5984e-dec2-4a98-9858-b6c9212c33f4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.698539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.702611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.704525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.705550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.707700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.710436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.739625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 827/1645] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:03.749931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.755196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.768887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.813786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc7e4b-3329-4759-9cc1-b63c3220c365 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.820660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 819/1646] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:03.821610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:03.824828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.825537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.825812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.826106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] PATH_INFO: `/v3/projects/0037bc62b05c4702b00c7a0a674220be/users/5219a5ebc792482cabdbefdbd574ca27/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.844193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4a05e-7e9a-4b11-83b8-acf702ebf981 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:03.858843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:03.859529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:03.859790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:03.860059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:03.861694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0037bc62b05c4702b00c7a0a674220be, user_id=5219a5ebc792482cabdbefdbd574ca27, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:03.864644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:03.870092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:03.893358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 828/1647] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:03 2022] PUT /identity/v3/projects/0037bc62b05c4702b00c7a0a674220be/users/5219a5ebc792482cabdbefdbd574ca27/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:03.897896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:04.017712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:04.023784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:04.048783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 820/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:04.076062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665eaa3f-eb62-4bd3-b413-cbd407a507b5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:04.091558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:04.092203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:04.092488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:04.093051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:04.093737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:04.096173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77959e2-9d85-4801-b541-ed93dead410e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:04.120034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 829/1649] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:04.120636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:04.142151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed930b9-4c47-4fcf-be33-b71a1e79198c service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:04.159213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:04.159929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:04.160743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:04.160743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] PATH_INFO: `/v3/projects/0037bc62b05c4702b00c7a0a674220be/users/5219a5ebc792482cabdbefdbd574ca27/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:04.167501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0037bc62b05c4702b00c7a0a674220be, user_id=5219a5ebc792482cabdbefdbd574ca27, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:04.177089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:04.217206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 821/1650] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:04 2022] PUT /identity/v3/projects/0037bc62b05c4702b00c7a0a674220be/users/5219a5ebc792482cabdbefdbd574ca27/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:04.221740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bee27c-f9cb-49fb-91ed-74810fdb2487 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:04.222064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bee27c-f9cb-49fb-91ed-74810fdb2487 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:04.222292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bee27c-f9cb-49fb-91ed-74810fdb2487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:04.302906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-12bee27c-f9cb-49fb-91ed-74810fdb2487 None None] MFA Rules not processed for user `5219a5ebc792482cabdbefdbd574ca27`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:04.304268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-12bee27c-f9cb-49fb-91ed-74810fdb2487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:04.550846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 830/1651] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:04.678401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:04.816806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab45d4-4b9c-4a6b-8657-a9130e494508 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:04.827327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:04.829823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:04.830272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:04.831092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:04.857013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:04.859499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0265f-d9b9-4910-b2b7-3c949cd6612a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:04.905233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 822/1652] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:05.053827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484efbd-e2a5-419e-8147-3259931e05bf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:05.054131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484efbd-e2a5-419e-8147-3259931e05bf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:05.054390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484efbd-e2a5-419e-8147-3259931e05bf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:05.057143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 831/1653] 10.222.0.46 () {54 vars in 962 bytes} [Thu Sep 1 07:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:05.079302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268c1f-4e05-4620-bd70-e1833f6baae6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:05.079859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268c1f-4e05-4620-bd70-e1833f6baae6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:05.079955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5268c1f-4e05-4620-bd70-e1833f6baae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:05.126660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d5268c1f-4e05-4620-bd70-e1833f6baae6 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:05.132860 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5268c1f-4e05-4620-bd70-e1833f6baae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:05.144889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:05.306145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 823/1654] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Sep 1 07:14:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:05.307363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd11bd-6b21-4486-a7b2-82cac61ac70d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:05.307832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd11bd-6b21-4486-a7b2-82cac61ac70d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:05.307976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd11bd-6b21-4486-a7b2-82cac61ac70d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:05.309832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 832/1655] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 07:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:05.316997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:05.317599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:05.318007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:05.366080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:05.368730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:05.397690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 824/1656] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:14:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:05.677245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd11bd-6b21-4486-a7b2-82cac61ac70d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:05.699906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd11bd-6b21-4486-a7b2-82cac61ac70d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:05.707538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:05.710780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:05.711258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:05.711816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:05.729418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:05.731101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3dbe4b-9f99-4c1e-b093-3c5ccc6d3d2a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:05.756555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 833/1657] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:07.322213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:07.335795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3143166-ece3-47fb-b5cb-630f5889223d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:07.341717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:07.342597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:07.342889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:07.343297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:07.461822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:07.464035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa847a0-9305-424f-8935-12a02c72171a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:07.504874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 825/1658] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:09.872938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:09.873455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:09.873685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:09.952496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] MFA Rules not processed for user `db4f0928502a43b490de01afc587a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:09.956838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:10.154395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:10.203873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 834/1659] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:10.217572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.218107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.218107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.312184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] MFA Rules not processed for user `db4f0928502a43b490de01afc587a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:10.318313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:10.342497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 826/1660] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:10.358657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:10.378661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:10.394669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f95a2-533d-4718-9ec3-ccba7d2b02a6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:10.410905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:10.411543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.411828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.416405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.453537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:10.455695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:10.496612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 835/1661] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:10.537613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:10.550732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3b9dd-2b61-4bba-b8af-c4cfc5a7d8ee service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:10.563090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:10.563593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.563882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.564641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.607039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7310cc11-77b8-4d52-87a1-ca2252651145 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:10.647731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:10.650071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.650915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.652716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] PATH_INFO: `/v3/users/2a306c3e658e4a52a8618cf48117953b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.655156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a306c3e658e4a52a8618cf48117953b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:10.662524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:10.738085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:10.739754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abe3e2-3585-45b9-a442-3bb174e4ceba service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:10.772886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 836/1662] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:10.777868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d095-5851-4ce4-9dce-1a104d2c556a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.778145 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d095-5851-4ce4-9dce-1a104d2c556a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.778424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d095-5851-4ce4-9dce-1a104d2c556a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.779431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:10.812828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a306c3e658e4a52a8618cf48117953b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:10.829750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a306c3e658e4a52a8618cf48117953b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:10.851759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a306c3e658e4a52a8618cf48117953b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:10.852562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a306c3e658e4a52a8618cf48117953b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:10.867405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b7f3d095-5851-4ce4-9dce-1a104d2c556a None None] MFA Rules not processed for user `561fa6572e0b4b949d1c09ee9e9d5e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:10.870655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b7f3d095-5851-4ce4-9dce-1a104d2c556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:10.886091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7eeda586-6322-4e4c-9bb7-80f29b45d027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a306c3e658e4a52a8618cf48117953b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:10.911505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 827/1663] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:10 2022] DELETE /identity/v3/users/2a306c3e658e4a52a8618cf48117953b => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:10.918004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:10.918414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:10.918786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:10.964570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:10.965843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.026698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 837/1664] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:11.039538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.039857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.040086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:11.081703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] MFA Rules not processed for user `561fa6572e0b4b949d1c09ee9e9d5e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:11.083863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.128577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 828/1665] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:11.157128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:11.159921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.285111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 838/1666] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:11.285111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:11.285111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.301559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:11.336504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609586b-36b2-4d49-a856-5fa937c40316 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:11.352291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:11.352920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.353304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.353580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:11.384560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:11.387496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83246a3d-d8b6-4911-9173-7c19937808ed service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:11.416562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 829/1667] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:11.483628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4bc17-3256-414e-b752-d23f5be20d64 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:11.496532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:11.497193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.498000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.498000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:11.518615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:11.520278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867ee30-189b-4928-9bfa-2984da54da33 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:11.543455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 839/1668] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:11.830514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.830796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.831073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:11.832445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 830/1669] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 07:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:11.838535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064f46-6d38-41dd-a97d-3cd5c073756b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.838824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064f46-6d38-41dd-a97d-3cd5c073756b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.839102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0064f46-6d38-41dd-a97d-3cd5c073756b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:11.905094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c0064f46-6d38-41dd-a97d-3cd5c073756b None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:11.906634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c0064f46-6d38-41dd-a97d-3cd5c073756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.920182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 840/1670] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:14:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:11.945257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:11.947108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:11.974835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d9971-4849-4d41-a46c-c98c364ff9c6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:11.982357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:11.984272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:11.984607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:11.984919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:12.005773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:12.007764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e23b5c-059a-41f9-bfd0-19140d7990aa service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:12.027779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 831/1671] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:12.294377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a2770-dfca-448d-a99c-ec977de4bd68 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:12.294927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a2770-dfca-448d-a99c-ec977de4bd68 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:12.294927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a2770-dfca-448d-a99c-ec977de4bd68 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:12.298573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 841/1672] 10.222.0.46 () {54 vars in 912 bytes} [Thu Sep 1 07:14:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:12.307759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:12.308298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:12.309002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:12.347581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:12.358044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:12.388203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 832/1673] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:14:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:12.417333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a2770-dfca-448d-a99c-ec977de4bd68 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:12.446318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a2770-dfca-448d-a99c-ec977de4bd68 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:12.459420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:12.461800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:12.465606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:12.466015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:12.490555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:12.492359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147e11c-80c6-429f-ac0a-278153536691 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:12.532177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 842/1674] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:12.595968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:12.602068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:12.628593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab025-405d-4ec8-b04e-fe073235bb2a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:12.637980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:12.638647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:12.638925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:12.639200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] PATH_INFO: `/v3/projects/4e94525a3f1c4bc5a6a8846e3a1d134e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:12.645968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e94525a3f1c4bc5a6a8846e3a1d134e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:12.648932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:14:12.648932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 01 07:14:12.648932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: Sep 01 07:14:12.650063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:12.736110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:12.857080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e94525a3f1c4bc5a6a8846e3a1d134e', 'request_id': 'req-0ee4f3cd-6c7b-43ca-927d-59df926b3382'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:12.899947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e94525a3f1c4bc5a6a8846e3a1d134e', 'request_id': 'req-0ee4f3cd-6c7b-43ca-927d-59df926b3382'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:12.924900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e94525a3f1c4bc5a6a8846e3a1d134e', 'request_id': 'req-0ee4f3cd-6c7b-43ca-927d-59df926b3382'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:12.960710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] The token cache is being invalidate because project 4e94525a3f1c4bc5a6a8846e3a1d134e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:12.962109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:12.971581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:12.981602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 833/1675] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:12 2022] DELETE /identity/v3/projects/4e94525a3f1c4bc5a6a8846e3a1d134e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:12.983670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6bbc-c049-456b-96a7-66390e0b84f4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:12.984120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6bbc-c049-456b-96a7-66390e0b84f4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:12.984494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6bbc-c049-456b-96a7-66390e0b84f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.047346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-af2d6bbc-c049-456b-96a7-66390e0b84f4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:13.047529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:13.048924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af2d6bbc-c049-456b-96a7-66390e0b84f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.049477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.285674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 843/1676] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:13.288504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.289106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.289950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.290505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4f3cd-6c7b-43ca-927d-59df926b3382 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:13.308027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:13.309288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.309794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.310277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.312520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.326092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:13.328080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06342eb6-938b-454c-956d-abbcc592404a service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:13.352813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 834/1677] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:13.352813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.352813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.352813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.366391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] MFA Rules not processed for user `db4f0928502a43b490de01afc587a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:13.372430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.429603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:13.430841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.479424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 835/1678] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:13.480616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:13.503846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896b5d-21e5-434f-ae57-ea4aab94459e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:13.529082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:13.530065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.530731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.531182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.532032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:13.532510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:13.534343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbda828-37f0-480e-9f2d-bf4274b9b7b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:13.582305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 844/1679] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:13.589390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 836/1680] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:13.610362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:13.646921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:13.660479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519586e0-1ffb-48b2-b014-cddbc3a2e9a5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:13.689806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:13.690625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.691376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.692873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.693798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:13.702593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:13.738535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.738816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.739133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.799212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 845/1681] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:13 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:13.803296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:13.820996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] MFA Rules not processed for user `2096dd4d4b6f49fc8572a4bcd9689248`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:13.829382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:13.955149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd6c79-b310-4dc3-a3e6-c9fae4b6f0d3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:13.990773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:13.991466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:13.992468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:13.992690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:13.993470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:13.995313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.045837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 837/1682] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:14.065719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:14.067899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:14.072919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 846/1683] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:13 2022] POST /identity/v3/users => generated 303 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:14.076944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:14.119774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a37cd0-8b4a-485d-8f3f-87b149ee8b2f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:14.127694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:14.138538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:14.140788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.144253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.144531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.148052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:14.152052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.206305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 847/1684] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:14.207294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:14.244808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01233-9f8d-436f-9693-5ac5fa45a4be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:14.273716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:14.274348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.274626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.275745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.279719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:14.291106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.295916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168484-57f2-4ef4-92f4-3c7d23b18654 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:14.311187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:14.311838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.314352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.314642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.326128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 848/1685] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:14.332181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:14.341409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:14.346474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e004a-a3b5-458d-a6f0-f95b8e161305 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.358406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63015d0-f5b4-45b8-8f13-e0a8f8039251 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:14.366315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:14.368057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.369244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.376221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] PATH_INFO: `/v3/projects/9e458a83bb4f486d9c5bac9a0bb35299/users/1caf1fd9b6294870993fb96c2214aad5/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.376874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 838/1686] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:14.406021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e458a83bb4f486d9c5bac9a0bb35299, user_id=1caf1fd9b6294870993fb96c2214aad5, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:14.410710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.429242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 849/1687] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:14 2022] PUT /identity/v3/projects/9e458a83bb4f486d9c5bac9a0bb35299/users/1caf1fd9b6294870993fb96c2214aad5/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:14.434192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34be152-0a87-48d3-a400-f39b16b9e0e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.434622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34be152-0a87-48d3-a400-f39b16b9e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.435169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34be152-0a87-48d3-a400-f39b16b9e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.478952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e34be152-0a87-48d3-a400-f39b16b9e0e3 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:14.480197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e34be152-0a87-48d3-a400-f39b16b9e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:14.627887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 839/1688] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:14.672916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:14.675470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:14.842464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9a3c9-4859-4ded-8572-714cee93e196 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:14.847865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:14.848970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:14.849326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:14.849607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:14.907665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:14.912041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:14.946028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 850/1689] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:17.298535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:17.299023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:17.299303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:17.382129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] MFA Rules not processed for user `1caf1fd9b6294870993fb96c2214aad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:17.384637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:17.512915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 840/1690] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:17.517838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:17.534531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c4b40-5f3e-429a-b5c5-4a325a7f4c27 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:17.547992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:17.550165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:17.552552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:17.553344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:17.555247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:17.560411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:17.634113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 851/1691] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:17 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:17.637896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:17.661966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36800d-d4af-4775-92af-8cec971ad16a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:17.723961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:17.827820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:17.828565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:17.828900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:17.829205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:17.829853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:17.832330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:17.940751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 841/1692] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:17 2022] POST /identity/v3/users => generated 308 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:17.945634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:17.963046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f04b8-6a6f-4b22-b619-0e6cdd13e757 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:17.988601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:17.989310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:17.989591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:17.989867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:17.990536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:17.992173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:17.993771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:18.018599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 852/1693] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:18.023843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:18.037762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97362e0-1a1b-4cd2-ac00-ebe6168b428f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:18.044996 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:18.045987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:18.047068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:18.047555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] PATH_INFO: `/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59/users/289e85cd306c4c82b9069558bccd86c2/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:18.094688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef578c0e6a9e4ba488d3e606c0f91b59, user_id=289e85cd306c4c82b9069558bccd86c2, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:18.097535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:18.145547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:18.151127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 842/1694] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:18 2022] PUT /identity/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59/users/289e85cd306c4c82b9069558bccd86c2/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:18.186866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9eaf3-f3be-4b0e-bdd1-fc5a417fb656 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:18.423867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:18.425520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:18.425819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:18.426099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:18.430509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:18.432092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c772762-232a-4701-8f5c-45cd141dc67a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:18.463811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 853/1695] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:18.468157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:18.484474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:18.485094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:18.485486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1e597-62c8-4193-8880-f5caf1bd9784 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:18.485949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:18.506282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:18.507011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:18.507374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:18.507753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] PATH_INFO: `/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59/users/289e85cd306c4c82b9069558bccd86c2/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:18.510664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef578c0e6a9e4ba488d3e606c0f91b59, user_id=289e85cd306c4c82b9069558bccd86c2, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:18.518820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c16c9d-d5b2-4e20-b8fb-eb109cec7d62 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:18.524025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] MFA Rules not processed for user `2096dd4d4b6f49fc8572a4bcd9689248`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:18.529163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:18.559589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 843/1696] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:18 2022] PUT /identity/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59/users/289e85cd306c4c82b9069558bccd86c2/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:18.675256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 854/1697] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:19.780252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:19.780711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:19.781083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:19.961561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:19.962807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:20.088180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 844/1698] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:20.314170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:20.315727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:20.705406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899b1e6-760e-46fc-8a49-51309b98da09 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:20.784828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:20.830770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:20.832345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:20.868037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:20.868667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:20.868941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:20.872728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:20.898556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:20.902038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:20.924112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 855/1699] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:14:20.981639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:20.983454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:21.012382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840a527-e6e6-43f8-8ef0-cdf2a18ea62e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:21.025990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:21.027005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:21.027478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:21.027977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:21.036948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:21.058003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:21.156882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cc24c-8882-490f-b991-b2bceda90e7a service swift] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:21.169670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:21.177813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:21.179039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:21.180114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] PATH_INFO: `/v3/users/9d28e747db8044e18ba2ce4a100cbaaa` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:21.182222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d28e747db8044e18ba2ce4a100cbaaa)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:21.183652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: WARNING py.warnings [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 07:14:21.183652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: warnings.warn(msg) Sep 01 07:14:21.183652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: Sep 01 07:14:21.185402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:21.290464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:21.292259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ca2b1-50d6-4455-a366-8f743a9fae78 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:21.309971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d28e747db8044e18ba2ce4a100cbaaa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:21.322540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d28e747db8044e18ba2ce4a100cbaaa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:21.328013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 845/1700] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:21.344142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d28e747db8044e18ba2ce4a100cbaaa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:21.345017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d28e747db8044e18ba2ce4a100cbaaa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:21.364640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d28e747db8044e18ba2ce4a100cbaaa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:21.437280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 856/1701] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:20 2022] DELETE /identity/v3/users/9d28e747db8044e18ba2ce4a100cbaaa => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:21.438931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959611c4-d3a4-4b7a-92ce-a0888e19dbdd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:21.439331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959611c4-d3a4-4b7a-92ce-a0888e19dbdd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:21.439714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959611c4-d3a4-4b7a-92ce-a0888e19dbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:21.486195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-959611c4-d3a4-4b7a-92ce-a0888e19dbdd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:21.492530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-959611c4-d3a4-4b7a-92ce-a0888e19dbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:21.589150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 846/1702] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:21.629250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:21.630970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:21.699130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:21.748105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6122a-4395-4532-8e7a-41e0af5b8bfb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:21.770646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:21.771749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:21.772037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:21.772311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:21.789885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:21.793100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:21.822588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 857/1703] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:22.435289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:22.435782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:22.435883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:22.533401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] MFA Rules not processed for user `289e85cd306c4c82b9069558bccd86c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:22.533401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:22.663125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:22.665550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:22.696483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872403a-ddd1-40dc-a499-e33c137fb964 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:22.710405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:22.711112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:22.711396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:22.714065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] PATH_INFO: `/v3/projects/5be5442e873f45e5a98a95b58a9a7ede` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:22.717379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5be5442e873f45e5a98a95b58a9a7ede)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:22.719604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:22.730108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 847/1704] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:22.854866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5be5442e873f45e5a98a95b58a9a7ede', 'request_id': 'req-48d7cc77-de96-4d4c-ba87-40224503fb3c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:22.873550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5be5442e873f45e5a98a95b58a9a7ede', 'request_id': 'req-48d7cc77-de96-4d4c-ba87-40224503fb3c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:22.885206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5be5442e873f45e5a98a95b58a9a7ede', 'request_id': 'req-48d7cc77-de96-4d4c-ba87-40224503fb3c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:22.915062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] The token cache is being invalidate because project 5be5442e873f45e5a98a95b58a9a7ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:22.915349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:22.916259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-48d7cc77-de96-4d4c-ba87-40224503fb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:22.922804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:22.924119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 858/1705] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:22 2022] DELETE /identity/v3/projects/5be5442e873f45e5a98a95b58a9a7ede => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:22.936836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f83de-f059-4ba2-afbc-fc6375522165 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:23.015022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:23.068453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:23.069446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:23.069875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:23.072140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] PATH_INFO: `/v3/users/5d7d310d4afa49f183e48aba44b2ac0f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:23.072662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7d310d4afa49f183e48aba44b2ac0f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:23.122522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:23.300094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7d310d4afa49f183e48aba44b2ac0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:23.329593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7d310d4afa49f183e48aba44b2ac0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:23.355586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7d310d4afa49f183e48aba44b2ac0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:23.358851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7d310d4afa49f183e48aba44b2ac0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:23.407550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7d310d4afa49f183e48aba44b2ac0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:23.430405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 848/1706] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:22 2022] DELETE /identity/v3/users/5d7d310d4afa49f183e48aba44b2ac0f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:23.508026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:23.508547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:23.509641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:23.539998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:23.543168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:23.698448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 859/1707] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 07:14:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:23.728240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:23.730510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:23.771490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:23.772980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:23.791237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69462b84-cfbc-4f58-b56e-a16130dd55e8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:23.811359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:23.814556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:23.816274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:23.818019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:23.818765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2749c1-d3a5-474b-9e7c-fca85e87fd56 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:23.841589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:23.847290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89706f40-554e-4661-a98a-f6d41b3c4a61 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:23.851314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:23.852715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:23.855038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:23.855331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:23.874256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 860/1708] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:23.890763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:23.907001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4e12c-4f18-48d1-8009-a6a7bebade2d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:23.966618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 849/1709] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:25.831248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229394a7-412e-4447-b766-79053972d67f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:25.831766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229394a7-412e-4447-b766-79053972d67f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:25.831855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229394a7-412e-4447-b766-79053972d67f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:25.834637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 861/1710] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:25.902622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757895-8b07-4f29-8e6d-6924c619c38a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:25.903523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757895-8b07-4f29-8e6d-6924c619c38a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:25.904139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db757895-8b07-4f29-8e6d-6924c619c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:25.938368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-db757895-8b07-4f29-8e6d-6924c619c38a None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:25.943303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db757895-8b07-4f29-8e6d-6924c619c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:26.162600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 850/1711] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:14:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:26.213080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41365fd-579f-47b7-8996-e38aeda40102 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:26.213668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41365fd-579f-47b7-8996-e38aeda40102 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:26.214140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41365fd-579f-47b7-8996-e38aeda40102 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:26.215646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 862/1712] 10.222.0.46 () {54 vars in 961 bytes} [Thu Sep 1 07:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:26.235818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:26.236160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:26.236369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:26.315877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:26.317985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:26.378069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 851/1713] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Sep 1 07:14:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:26.389770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41365fd-579f-47b7-8996-e38aeda40102 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:26.422349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:26.423760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:26.427983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41365fd-579f-47b7-8996-e38aeda40102 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:26.460763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:26.465345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:26.465770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:26.475103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:26.527423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:26.529064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:26.565252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:26.584649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 863/1714] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:26.587700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:26.591215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:26.664319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1aa48-1db8-46d4-812a-3e00d12b88e0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:26.675833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:26.676995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:26.677322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:26.678120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] PATH_INFO: `/v3/users/5af41c1833234fa1985029653ef6d320` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:26.680501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af41c1833234fa1985029653ef6d320)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:26.682515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:26.822445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af41c1833234fa1985029653ef6d320'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:26.834971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:26.845811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af41c1833234fa1985029653ef6d320'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:26.871921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af41c1833234fa1985029653ef6d320'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:26.873398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af41c1833234fa1985029653ef6d320'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:26.894615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f8b3-bc23-47cb-8be9-4f3469ff524d service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:26.907614 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:26.908669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:26.909203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:26.911057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] PATH_INFO: `/v3/users/c729490faaa643ffbbdb6e5178b28dd2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:26.916009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c729490faaa643ffbbdb6e5178b28dd2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:26.919413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:26.922010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af41c1833234fa1985029653ef6d320'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:26.946075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 852/1715] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:26 2022] DELETE /identity/v3/users/5af41c1833234fa1985029653ef6d320 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:26.946075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:26.946075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.031476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c748b-1f3f-4b51-a531-e2b299b7b007 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.037779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.038420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.038703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.039152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] PATH_INFO: `/v3/projects/1c1ce987708144a69d878f1fd401a7d5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.040312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c1ce987708144a69d878f1fd401a7d5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.042333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.082919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c729490faaa643ffbbdb6e5178b28dd2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.083939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c729490faaa643ffbbdb6e5178b28dd2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.093591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c729490faaa643ffbbdb6e5178b28dd2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.094536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c729490faaa643ffbbdb6e5178b28dd2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.103881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c1ce987708144a69d878f1fd401a7d5', 'request_id': 'req-5aa2fe17-0402-42c1-a434-d981b26fd678'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.104782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9206b556-2312-4cfe-9c8d-a6f8e462ce99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c729490faaa643ffbbdb6e5178b28dd2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.105964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:27.113185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c1ce987708144a69d878f1fd401a7d5', 'request_id': 'req-5aa2fe17-0402-42c1-a434-d981b26fd678'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.121665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c1ce987708144a69d878f1fd401a7d5', 'request_id': 'req-5aa2fe17-0402-42c1-a434-d981b26fd678'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.137926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 864/1716] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:26 2022] DELETE /identity/v3/users/c729490faaa643ffbbdb6e5178b28dd2 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:27.139737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc21dd-086d-4590-9aa9-3a41f3115fa0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.140268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc21dd-086d-4590-9aa9-3a41f3115fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.140566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc21dd-086d-4590-9aa9-3a41f3115fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.143751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] The token cache is being invalidate because project 1c1ce987708144a69d878f1fd401a7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:27.144060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.145698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5aa2fe17-0402-42c1-a434-d981b26fd678 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.148592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 853/1717] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:26 2022] DELETE /identity/v3/projects/1c1ce987708144a69d878f1fd401a7d5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:27.150076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3184bf-d1e7-453a-93a8-d3674cce40a7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.150403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3184bf-d1e7-453a-93a8-d3674cce40a7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.150684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3184bf-d1e7-453a-93a8-d3674cce40a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.179702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da3184bf-d1e7-453a-93a8-d3674cce40a7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:27.180971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da3184bf-d1e7-453a-93a8-d3674cce40a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.189721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-75cc21dd-086d-4590-9aa9-3a41f3115fa0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:27.191740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-75cc21dd-086d-4590-9aa9-3a41f3115fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.293191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 865/1718] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:27.296666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4947d4-efb1-4a8e-9d49-7d4159b116e7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.297224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4947d4-efb1-4a8e-9d49-7d4159b116e7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.297685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4947d4-efb1-4a8e-9d49-7d4159b116e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.311907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 854/1719] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:27.314410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.314410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.314410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.315916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 855/1720] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:27.330999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd4947d4-efb1-4a8e-9d49-7d4159b116e7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:27.333055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd4947d4-efb1-4a8e-9d49-7d4159b116e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.341488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 866/1721] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:27.350015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.351661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16a2f8-f154-4fcd-ab1f-dad6003f1d13 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.351960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16a2f8-f154-4fcd-ab1f-dad6003f1d13 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.352073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.352241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16a2f8-f154-4fcd-ab1f-dad6003f1d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.376545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1e16a2f8-f154-4fcd-ab1f-dad6003f1d13 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:27.377832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1e16a2f8-f154-4fcd-ab1f-dad6003f1d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.397103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 867/1722] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:27.399191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.399612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.400402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.430010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:27.431245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.450999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf0f7-957f-469b-84a2-3372ea9ceb19 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.456560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.458245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.459066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.459641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.480486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.482120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.493933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 856/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:27.495826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.503158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:27.517406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4346fb7-622e-49ff-9c53-9532fc3eedfd service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.528609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.530629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.531089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.531506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.542808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.543319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 868/1724] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:27.544277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.545386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.558541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 857/1725] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:27.562255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba155f1-d00d-4cbf-b4e2-78a92b46206d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.568420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.569049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.569999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.569999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.570420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:27.570824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.572468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.572943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.574984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.591344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 869/1726] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:27.592731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.596216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a1b8a-94b6-497c-b452-a83257b098e0 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.603859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.604362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.604643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.604975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.606505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b700e89-a8b5-4915-b799-420712eb752d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.613299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.614266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.614749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.615219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.616489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.616735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.618009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.619300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.631834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 858/1727] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:27.659610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 870/1728] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:27.660007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.682892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db260c-480c-424d-92b8-902818f1ff10 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.804179 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:27.818267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.819565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.820129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.820827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.821783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.828705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.889152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 859/1729] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:27 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:27.892943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.893623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.896330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:27.907553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943587c-93c5-419d-a140-a4c505f391ad admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.909570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd83e42-6788-498f-8f97-561268f48819 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.914677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.915367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.915693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.915972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.916395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.916965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.917488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.917965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.918565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] PATH_INFO: `/v3/users/1caf1fd9b6294870993fb96c2214aad5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.919902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.920139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1caf1fd9b6294870993fb96c2214aad5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.922859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.934982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 860/1730] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:27.940155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:27.967044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587b0c-4d76-401e-8a72-34e4f4a67e56 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:27.976318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:27.978847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:27.979185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:27.979402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:27.980274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:27.993077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:27.997163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1caf1fd9b6294870993fb96c2214aad5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:27.999041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1caf1fd9b6294870993fb96c2214aad5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.011830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1caf1fd9b6294870993fb96c2214aad5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.012752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1caf1fd9b6294870993fb96c2214aad5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.012979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 861/1731] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:28.018262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.021949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.025023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2507dbb-3ea3-4fa5-b504-2bb96ea80a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1caf1fd9b6294870993fb96c2214aad5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.047647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6b767-0769-40e3-8b6c-c0d08ca6a79c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.057085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.057881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.058654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.059133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] PATH_INFO: `/v3/projects/b58647e15fa343b8b75bf51e7f0792ab/users/ecaea71e52ca46fca7dc57c50057f2c7/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.084028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58647e15fa343b8b75bf51e7f0792ab, user_id=ecaea71e52ca46fca7dc57c50057f2c7, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.088714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 871/1732] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:27 2022] DELETE /identity/v3/users/1caf1fd9b6294870993fb96c2214aad5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.088714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f458a-8a7b-4ab8-8796-e2f56f4d7d10 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.088943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f11ac4-2a97-43cb-817b-5207a2c1861e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.089025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f458a-8a7b-4ab8-8796-e2f56f4d7d10 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.089025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f458a-8a7b-4ab8-8796-e2f56f4d7d10 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.089447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 872/1733] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:28.103563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.103849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.104408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.115677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 862/1734] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:28 2022] PUT /identity/v3/projects/b58647e15fa343b8b75bf51e7f0792ab/users/ecaea71e52ca46fca7dc57c50057f2c7/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.119189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.119489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.120124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.138148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:28.139257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.178767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] MFA Rules not processed for user `ecaea71e52ca46fca7dc57c50057f2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:28.180611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.222616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 873/1735] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:28.224872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.226501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.235177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:28.307799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 863/1736] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:28.307799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.308528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.332785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c50d-1839-41ac-be74-394c3e087ae5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.335038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81f552-2cbc-421a-b8c9-92ffa24fe299 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.339259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.340656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.340943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.342069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] PATH_INFO: `/v3/projects/917622d952994fae9715d8c50809324a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.343545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917622d952994fae9715d8c50809324a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.344648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.345306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.345584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.345698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.345853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.393786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.396748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.416645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 864/1737] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:28.417385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.430053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72409c93-4fc9-45a1-ac9b-92be0aa14e70 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.435969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.436933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.437445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.437801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.438681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.441063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.458786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917622d952994fae9715d8c50809324a', 'request_id': 'req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.486597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917622d952994fae9715d8c50809324a', 'request_id': 'req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.496848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917622d952994fae9715d8c50809324a', 'request_id': 'req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.504963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 865/1738] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:28 2022] POST /identity/v3/projects => generated 393 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:28.510432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.517325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.518629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] The token cache is being invalidate because project 917622d952994fae9715d8c50809324a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:28.519356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.520155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.525726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 874/1739] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:28 2022] DELETE /identity/v3/projects/917622d952994fae9715d8c50809324a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.526924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.528510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.589908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:28.616113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c77cab-bba6-4036-a6d5-0b4c6ef65f11 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.620936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.621706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.622012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.622287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] PATH_INFO: `/v3/projects/9e458a83bb4f486d9c5bac9a0bb35299` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.623408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e458a83bb4f486d9c5bac9a0bb35299)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.625350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.655927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319e5b-4cd6-4669-a6e0-19fc11d41fc4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.663397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.664235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.664639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.664911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] PATH_INFO: `/v3/projects/c4f0d85a8fee4b2a95d2c05d30afc078` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.666095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f0d85a8fee4b2a95d2c05d30afc078)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.668840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.684985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e458a83bb4f486d9c5bac9a0bb35299', 'request_id': 'req-31ed659a-f68d-4014-9a3b-a5e81903365f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.691674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e458a83bb4f486d9c5bac9a0bb35299', 'request_id': 'req-31ed659a-f68d-4014-9a3b-a5e81903365f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.697660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e458a83bb4f486d9c5bac9a0bb35299', 'request_id': 'req-31ed659a-f68d-4014-9a3b-a5e81903365f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.710548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:28.712925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] The token cache is being invalidate because project 9e458a83bb4f486d9c5bac9a0bb35299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:28.713277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.714097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.718727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 866/1740] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:28 2022] DELETE /identity/v3/projects/9e458a83bb4f486d9c5bac9a0bb35299 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.718727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.718727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.729865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f0d85a8fee4b2a95d2c05d30afc078', 'request_id': 'req-a0f0171f-005a-4268-ab3a-64b629713c8c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.740069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f0d85a8fee4b2a95d2c05d30afc078', 'request_id': 'req-a0f0171f-005a-4268-ab3a-64b629713c8c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.759043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f0d85a8fee4b2a95d2c05d30afc078', 'request_id': 'req-a0f0171f-005a-4268-ab3a-64b629713c8c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.787199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] The token cache is being invalidate because project c4f0d85a8fee4b2a95d2c05d30afc078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:28.787553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.788383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.790272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 875/1741] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:28 2022] DELETE /identity/v3/projects/c4f0d85a8fee4b2a95d2c05d30afc078 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.791366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.792725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.805915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed659a-f68d-4014-9a3b-a5e81903365f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.861207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0171f-005a-4268-ab3a-64b629713c8c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.864813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.865451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.865727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.865978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.866081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.866607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.866703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.866885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.867159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] PATH_INFO: `/v3/users/a1303a49a7bf48c0b110e10f20bc59e1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.868435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1303a49a7bf48c0b110e10f20bc59e1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.868600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.870428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.895442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 867/1742] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:28 2022] POST /identity/v3/users => generated 303 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:28.895442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:28.897093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:28.909975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5344988-9195-4f17-b726-83cfc520c2e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:28.915976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:28.916609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.916907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.917315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] PATH_INFO: `/v3/users/289e85cd306c4c82b9069558bccd86c2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.918451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289e85cd306c4c82b9069558bccd86c2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:28.920333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:28.927492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1303a49a7bf48c0b110e10f20bc59e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.929527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1303a49a7bf48c0b110e10f20bc59e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.936937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1303a49a7bf48c0b110e10f20bc59e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.937662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:28.938112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1303a49a7bf48c0b110e10f20bc59e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.947129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a743083-b4b3-4d00-9920-359ab48f6c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1303a49a7bf48c0b110e10f20bc59e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.974673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 876/1743] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:28 2022] DELETE /identity/v3/users/a1303a49a7bf48c0b110e10f20bc59e1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:28.976216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385f56-f2a6-4de5-96e2-d2057a24ae2c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:28.976545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385f56-f2a6-4de5-96e2-d2057a24ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:28.976820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385f56-f2a6-4de5-96e2-d2057a24ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:28.981825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289e85cd306c4c82b9069558bccd86c2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.989267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289e85cd306c4c82b9069558bccd86c2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.998801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289e85cd306c4c82b9069558bccd86c2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:28.999578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289e85cd306c4c82b9069558bccd86c2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.003160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6d385f56-f2a6-4de5-96e2-d2057a24ae2c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:29.005045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d385f56-f2a6-4de5-96e2-d2057a24ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.018839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289e85cd306c4c82b9069558bccd86c2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.027438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 868/1744] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:28 2022] DELETE /identity/v3/users/289e85cd306c4c82b9069558bccd86c2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:29.028826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.030611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.058969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:29.089165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 877/1745] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:29.096484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.097101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.097569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.116760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c947de-4113-4c57-b742-b53c333af3c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.123356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.124790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.125458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.126815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.127712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.130246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:29.131653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.131840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.142928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 878/1746] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:29.150360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.151330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 869/1747] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:29.156188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.162716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb325aa0-25e2-4acc-bede-1b95ecfd0a5a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.173182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.174157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.175169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.175169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.175304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f5357-5cd1-4fc9-9ca4-7946f3af8778 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.175591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:29.175903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.178478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.182224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.183091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.183377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.183656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] PATH_INFO: `/v3/projects/d48530e5d0a1470aa2ee149581710114/users/5f214fd3010c4e1c96ea6a281bf42524/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.192994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 879/1748] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:29.197626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.200928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48530e5d0a1470aa2ee149581710114, user_id=5f214fd3010c4e1c96ea6a281bf42524, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.203657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.209551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f74e-b1b8-462c-b904-84bdaa143c97 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.215750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.216304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.216683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.217032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.217833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.220433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.223407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 870/1749] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:29 2022] PUT /identity/v3/projects/d48530e5d0a1470aa2ee149581710114/users/5f214fd3010c4e1c96ea6a281bf42524/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:29.224720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.257220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 880/1750] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:29 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:29.261714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.323233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe22aa-17d1-458a-ab39-0e510a9cc62a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.332183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:29.340479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500b4f4-57d3-4d41-a55e-380a99485dd8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.346680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.347554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.348101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.348987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.349759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.351874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.387192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 881/1751] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:29 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:29.388112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.390518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.413518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4fb390-013d-4885-a206-61699e441652 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.415141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.415764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.416041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.416317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.416960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.419040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.419154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.419752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.420013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.420358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] PATH_INFO: `/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.421725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef578c0e6a9e4ba488d3e606c0f91b59)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.423735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.430368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 871/1752] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:29.431211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.441264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78c70f-b6db-4f92-a736-6009d0b31062 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.446350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.446981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.447248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.447524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.448209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.449855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.461575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 872/1753] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:29.462580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.465427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.475393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef578c0e6a9e4ba488d3e606c0f91b59', 'request_id': 'req-32b0d047-038e-4041-aa73-13b2fe1d81b3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.478426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a57f7-ee38-4409-977b-a28dec37a7ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.484199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef578c0e6a9e4ba488d3e606c0f91b59', 'request_id': 'req-32b0d047-038e-4041-aa73-13b2fe1d81b3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.490802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef578c0e6a9e4ba488d3e606c0f91b59', 'request_id': 'req-32b0d047-038e-4041-aa73-13b2fe1d81b3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.502852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] The token cache is being invalidate because project ef578c0e6a9e4ba488d3e606c0f91b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:29.503147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.503797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.506128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 882/1754] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:29 2022] DELETE /identity/v3/projects/ef578c0e6a9e4ba488d3e606c0f91b59 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:29.506865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.508127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.527153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:29.548148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.548836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.549069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.549381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] PATH_INFO: `/v3/projects/09a310c9dba84fce87f63338e2ada969` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.551746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a310c9dba84fce87f63338e2ada969)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.553727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.592114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0d047-038e-4041-aa73-13b2fe1d81b3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.601332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.602173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.602461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.602739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] PATH_INFO: `/v3/projects/d48530e5d0a1470aa2ee149581710114/users/5f214fd3010c4e1c96ea6a281bf42524/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.605050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48530e5d0a1470aa2ee149581710114, user_id=5f214fd3010c4e1c96ea6a281bf42524, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.608048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.624609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a310c9dba84fce87f63338e2ada969', 'request_id': 'req-a7c3389b-161c-4679-82da-d58227cbb81a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.628779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 883/1755] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:29 2022] PUT /identity/v3/projects/d48530e5d0a1470aa2ee149581710114/users/5f214fd3010c4e1c96ea6a281bf42524/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:29.639679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.639805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a310c9dba84fce87f63338e2ada969', 'request_id': 'req-a7c3389b-161c-4679-82da-d58227cbb81a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.639805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a310c9dba84fce87f63338e2ada969', 'request_id': 'req-a7c3389b-161c-4679-82da-d58227cbb81a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.643751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.674781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] The token cache is being invalidate because project 09a310c9dba84fce87f63338e2ada969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:29.675447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.676946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a7c3389b-161c-4679-82da-d58227cbb81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:29.678869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 873/1756] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:29 2022] DELETE /identity/v3/projects/09a310c9dba84fce87f63338e2ada969 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:29.682402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.682402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.682731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.728744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:29.730378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.762416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf30894-3436-4a06-9fc6-062e7e1a52b5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.766668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:29.769384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.771119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.771119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.771119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.771736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.773478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7668b86-9dde-46ef-90db-5b589b0eb0e9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.790827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 884/1757] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:29.795844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.796349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.796708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.843373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] MFA Rules not processed for user `5f214fd3010c4e1c96ea6a281bf42524`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:29.844771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.882578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 874/1758] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:29.883647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.889001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:29.904174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c3ee9-33ca-424a-a596-f8e901d2f28b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:29.909657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:29.910420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:29.911334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:29.911706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] PATH_INFO: `/v3/users/5219a5ebc792482cabdbefdbd574ca27` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:29.913135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5219a5ebc792482cabdbefdbd574ca27)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:29.915813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:29.983204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 885/1759] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:29.984119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:29.995307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.031507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5219a5ebc792482cabdbefdbd574ca27'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:30.037135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839706f-8c39-4862-be1b-cc1f6d6728e8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.048124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5219a5ebc792482cabdbefdbd574ca27'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:30.053087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.053835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.055735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.056083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] PATH_INFO: `/v3/projects/0a5e5a042d014387b27347c02f04e865/users/aafc4866f6a4414bb4a82cbe66f773b6/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.063649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5219a5ebc792482cabdbefdbd574ca27'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:30.064835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5219a5ebc792482cabdbefdbd574ca27'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:30.095541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-39152242-46b4-4961-9978-8afc4e2c4747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5219a5ebc792482cabdbefdbd574ca27'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:30.109821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5e5a042d014387b27347c02f04e865, user_id=aafc4866f6a4414bb4a82cbe66f773b6, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.114251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 875/1760] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:29 2022] DELETE /identity/v3/users/5219a5ebc792482cabdbefdbd574ca27 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:30.114873 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.115670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed1d9-faa8-44a7-911d-2b736419c43f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.115959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed1d9-faa8-44a7-911d-2b736419c43f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.116238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed1d9-faa8-44a7-911d-2b736419c43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.142486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 886/1761] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:29 2022] PUT /identity/v3/projects/0a5e5a042d014387b27347c02f04e865/users/aafc4866f6a4414bb4a82cbe66f773b6/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:30.153309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bafed1d9-faa8-44a7-911d-2b736419c43f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:30.153832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.154529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bafed1d9-faa8-44a7-911d-2b736419c43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.160121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.189472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:30.286331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7ed0d-364a-4b1d-97a8-95d1918c5830 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.287960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 876/1762] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:30.289916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.290241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.290503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.303254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.304002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.304280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.304560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.307561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.339006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] MFA Rules not processed for user `aafc4866f6a4414bb4a82cbe66f773b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:30.340542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.385704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:30.444052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.445635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.470126 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 887/1763] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:30 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:30.471139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.474526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 877/1764] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:30.480394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.486385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.502908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33fa26-0f07-45d9-96fc-5b8e2405c62f service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.515431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03671b-3ea1-4a05-b283-be6836462178 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.525757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.526495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.526965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.527249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.527913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:30.528219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.530930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.531977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.532438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.532592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.533030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.533865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.540143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.594538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 888/1765] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:30.594538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.613964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ea0de-b274-43e4-ada7-f2f348f5e0c6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.624710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.625303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.626328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.632665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.635285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 878/1766] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:30.639587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.639801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.641489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.708673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 889/1767] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:30.710866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.816446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58192349-a3e3-4aea-a41c-64b3b567f64d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.821791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c27fa-fe7d-4246-abac-c60d81651a57 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.823451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.824250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.824688 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.825111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.826105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.828285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.869533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 890/1768] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:30.870940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.883398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2111ee8-a011-4bd0-a276-ef60f696582c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:30.888960 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.889813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.890259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.890701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.891619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:30.893921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:30.897913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:30.899771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:30.900131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:30.900522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:30.904002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.906843 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:30.932064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 891/1769] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:30 2022] POST /identity/v3/users => generated 286 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:30.935621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:30.937989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:30.977083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37402101-9ca5-42b6-a1d1-03c7331785ff admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.001538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.003816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.004816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.005281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] PATH_INFO: `/v3/projects/0037bc62b05c4702b00c7a0a674220be` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.009107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0037bc62b05c4702b00c7a0a674220be)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.013801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.056308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.058365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.073822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 879/1770] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:31.075279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.082256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0037bc62b05c4702b00c7a0a674220be', 'request_id': 'req-dd5e63b1-f97f-4664-87d7-b106f08ea91a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:31.088041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd3c5-8f4a-4504-beb4-988c702234ba service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.093246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0037bc62b05c4702b00c7a0a674220be', 'request_id': 'req-dd5e63b1-f97f-4664-87d7-b106f08ea91a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:31.102124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0037bc62b05c4702b00c7a0a674220be', 'request_id': 'req-dd5e63b1-f97f-4664-87d7-b106f08ea91a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:31.122365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:31.143763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] The token cache is being invalidate because project 0037bc62b05c4702b00c7a0a674220be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:31.143763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:31.146662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:31.149998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 892/1771] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:30 2022] DELETE /identity/v3/projects/0037bc62b05c4702b00c7a0a674220be => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:31.150827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.157264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:31.234875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.235647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.235933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.236209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.236937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.239368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe092eb4-35cd-4cf1-aa91-75cebf6a7652 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.256246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 880/1772] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:31.258248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.258248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.258444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.290464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e63b1-f97f-4664-87d7-b106f08ea91a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.296004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.296004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.296004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.296004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.296648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.299672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.321090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:31.322164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:31.333903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 893/1773] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:31.335022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.343411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:31.355654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 881/1774] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:31.357787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.361284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde0986-ab4d-414a-be0e-ca8b4729c0d7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.369103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.370039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.370393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.370675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] PATH_INFO: `/v3/projects/546eff2970f345eca27610b6418d0924/users/32323304a230402b95b5865ea8facf6c/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.391201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546eff2970f345eca27610b6418d0924, user_id=32323304a230402b95b5865ea8facf6c, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.394504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fee35-c88e-43cb-a248-16eb7ad0a0d8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.397053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916220b-6043-4757-8b30-7d06a0fcb18e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.416989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.417721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.418005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.420470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 894/1775] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:31 2022] PUT /identity/v3/projects/546eff2970f345eca27610b6418d0924/users/32323304a230402b95b5865ea8facf6c/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:31.424869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.425174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.425455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.433697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.434794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.436677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.451175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:31.453061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:31.454541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 882/1776] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:31.458772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.479828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e617a2-1a07-49c5-9caf-92999eb27030 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.553294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:31.603056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 895/1777] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:31.604343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.648420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d02dd-a240-46e3-ab5f-a9395400807d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.659925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.662252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.663550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.663550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] PATH_INFO: `/v3/projects/90a16e5588df4f79905c64da5f3f5a7d/users/b83d5a243fd64e4e910800be64135b3a/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.684976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.686313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.686699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.687708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.689154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.691422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.708948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 883/1778] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:31.709156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a16e5588df4f79905c64da5f3f5a7d, user_id=b83d5a243fd64e4e910800be64135b3a, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.709265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.709473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.720573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e4fc8-f102-4c6e-aeaf-7aa88360b896 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.725385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.726019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.726293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.726659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.727182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:31.727629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.729321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.743760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 884/1779] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:31.744677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.775323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0c430-bf14-4532-a3f0-fd9bb62e3bd6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.776322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 896/1780] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:31 2022] PUT /identity/v3/projects/90a16e5588df4f79905c64da5f3f5a7d/users/b83d5a243fd64e4e910800be64135b3a/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:31.777568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:31.851406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.852398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.853247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.853683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] PATH_INFO: `/v3/projects/546eff2970f345eca27610b6418d0924/users/32323304a230402b95b5865ea8facf6c/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.856543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546eff2970f345eca27610b6418d0924, user_id=32323304a230402b95b5865ea8facf6c, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.858760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfa10f-da56-49c3-bc95-d08f21821d53 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:31.859513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3bce8-d758-49cf-a7e8-54b908b2b2d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.868590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:31.870956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.871432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.871785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.872627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:31.874841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e3c8a-5b89-4e8b-be9f-0f66d3ff5c25 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:31.885611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 885/1781] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:31 2022] PUT /identity/v3/projects/546eff2970f345eca27610b6418d0924/users/32323304a230402b95b5865ea8facf6c/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:31.887595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.887765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.888045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.920329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:31.921833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 897/1782] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:31 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:31.922730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:31.923009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:31.923290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:31.928078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] MFA Rules not processed for user `b83d5a243fd64e4e910800be64135b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:31.930489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:31.982611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] MFA Rules not processed for user `32323304a230402b95b5865ea8facf6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:31.983862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:32.084510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 886/1783] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:32.085139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.141599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 898/1784] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:32.147535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.182169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b51a6-9e4d-4c4e-96d8-bdda674aea97 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.188176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.189554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.189909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.190363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.191449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.194376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.211937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:32.248937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2561eaf-0a9c-4c78-8d84-3683a776d8de None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.258023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 887/1785] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:32 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:32.259522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.260175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.261078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.261658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.262123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.262814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:32.263210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.265151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.347184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 899/1786] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:32 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:32.347184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.418840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c05169-03e1-46b0-bb4b-9cc5a7e5bcdb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.484340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308488cf-5406-4985-bf47-4167c3b891a7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.490045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.490665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.490994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.491320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.491919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.493588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.505616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 900/1787] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:32.506719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.515890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:32.538321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a29e3-9b49-40ae-933e-7137a66116c4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.543743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.545528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.545528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.546625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.547839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.548535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.548829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.549142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.549934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.552790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.584935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 901/1788] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:32 2022] POST /identity/v3/users => generated 291 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:32.585812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.598726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae04a-aa03-44f3-a2f0-eacf48bcb63a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.605338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.605970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.606264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.606552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.607295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.609055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.641041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 902/1789] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:32.642852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.661501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd1dca-1850-4943-8441-6117dcfc23f5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.674011 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.674630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.675104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.675620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.676659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.676789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.677657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.679466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.690088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 888/1790] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:32.691367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.697545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 903/1791] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:32.701541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.705557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6bdc-e30b-4bc6-92c7-7be4b6f74084 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.714618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.715447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.716019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.716297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] PATH_INFO: `/v3/projects/d2ccdd669c33405b8435b26d611dd182/users/2db9026afdbd458a8b14757279c92917/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.733524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d2c7b-337b-4b76-91d6-caf2b681454b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.742379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.743970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.744357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.744849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] PATH_INFO: `/v3/projects/85b61f8d81f144d28598ebba6b084b78/users/25da6cbfd5d44cad88cf8c31610d3cf9/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.775750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ccdd669c33405b8435b26d611dd182, user_id=2db9026afdbd458a8b14757279c92917, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.783878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2c01b-c1fa-4d12-8c58-5cae61d04a60 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.785442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b61f8d81f144d28598ebba6b084b78, user_id=25da6cbfd5d44cad88cf8c31610d3cf9, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.797727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.807564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 889/1792] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:32 2022] PUT /identity/v3/projects/d2ccdd669c33405b8435b26d611dd182/users/2db9026afdbd458a8b14757279c92917/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:32.820794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.822654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.822954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.833999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 904/1793] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:32 2022] PUT /identity/v3/projects/85b61f8d81f144d28598ebba6b084b78/users/25da6cbfd5d44cad88cf8c31610d3cf9/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:32.837776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:32.895843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] MFA Rules not processed for user `2db9026afdbd458a8b14757279c92917`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:32.899085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:32.933902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:32.955830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e019f54-964d-4ddf-8575-c3c796763296 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:32.969176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:32.970271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:32.970805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:32.971245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:32.972080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:32.974196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:32.992193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 905/1794] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:32.993268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.037706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 890/1795] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:33.037706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.058267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae46754-932a-4cda-a53f-9de6fe824b8d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.068373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.069166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.069456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.069734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] PATH_INFO: `/v3/projects/85b61f8d81f144d28598ebba6b084b78/users/25da6cbfd5d44cad88cf8c31610d3cf9/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.072807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b61f8d81f144d28598ebba6b084b78, user_id=25da6cbfd5d44cad88cf8c31610d3cf9, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.090159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.132234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 891/1796] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:33 2022] PUT /identity/v3/projects/85b61f8d81f144d28598ebba6b084b78/users/25da6cbfd5d44cad88cf8c31610d3cf9/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:33.134536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.155655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1e501-a963-4564-b471-8434b302c19f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.210109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:33.236078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be9b83-3ab0-4866-ad12-7878bff73fbc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.236333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.236769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.237053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.237361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.239652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:33.243917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.244713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.244994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.245307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.245942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.247893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db898c6-3813-45c4-aafb-e94012516e15 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.288432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 892/1797] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:33 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:33.288432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.288432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.288432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.317988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.319418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.325218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] MFA Rules not processed for user `25da6cbfd5d44cad88cf8c31610d3cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:33.326915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:33.337266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 906/1798] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:33.338790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.355943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7971aa-552d-4d84-b702-d8d5a22f65c1 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.440109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 893/1799] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:33.447881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.448347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.448642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.448913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.449751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.451656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.465487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:33.471781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.490556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 907/1800] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:33 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:33.491521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.504234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa3c5c-6613-4067-9b1b-9e4ef3beda9f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.509391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.510010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.510294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.510598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.511251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.512857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.523479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 908/1801] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:33.527965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.540105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f2fb9-8687-4652-b372-9d191d2125f2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.545259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.545916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.546402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.546677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] PATH_INFO: `/v3/projects/260cfdfe5f43452ba4f53410c20f54f4/users/c5f443ea312841f1928d271e5d2d50a4/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.550687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ab213-dbb0-4cf5-b79e-674fa7573417 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.555076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.555819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.556559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.556559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.562835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260cfdfe5f43452ba4f53410c20f54f4, user_id=c5f443ea312841f1928d271e5d2d50a4, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.565576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.571915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.573858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.585244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 909/1802] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:33 2022] PUT /identity/v3/projects/260cfdfe5f43452ba4f53410c20f54f4/users/c5f443ea312841f1928d271e5d2d50a4/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:33.589550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.606064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 894/1803] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:33.670582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d0e64-9c91-4851-bbf1-50ae0dea55f8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.676920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.677647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.678207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.678369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.680436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.681996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce62a7-f819-4c92-89d6-6161992cbafa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.703247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 910/1804] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:33.707393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:33.719763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdeda1-f013-41c2-86d8-0dbfac82c338 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:33.726099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:33.727294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.727294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.727564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] PATH_INFO: `/v3/projects/260cfdfe5f43452ba4f53410c20f54f4/users/c5f443ea312841f1928d271e5d2d50a4/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.729866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=260cfdfe5f43452ba4f53410c20f54f4, user_id=c5f443ea312841f1928d271e5d2d50a4, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:33.733754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:33.760475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 895/1805] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:33 2022] PUT /identity/v3/projects/260cfdfe5f43452ba4f53410c20f54f4/users/c5f443ea312841f1928d271e5d2d50a4/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:33.776520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:33.776520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:33.776520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:33.813613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] MFA Rules not processed for user `c5f443ea312841f1928d271e5d2d50a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:33.814852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:33.920800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 911/1806] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:33.950127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.007570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38673cda-7f14-4607-9807-49be8f27a0de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.016443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.017212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.017490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.017760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.025468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:34.030413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.032534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:34.043172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 896/1807] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:34.140819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.185090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.185985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:34.211944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9395eb-38fe-4a3b-9757-f1e3af045dc2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.216936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.217630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.217916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.218194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] PATH_INFO: `/v3/users/b83d5a243fd64e4e910800be64135b3a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.219322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83d5a243fd64e4e910800be64135b3a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.221278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:34.256733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25062f75-612e-4c78-b2c7-070d71a305ee service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.262146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.262873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.263160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.263436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] PATH_INFO: `/v3/users/2db9026afdbd458a8b14757279c92917` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.264812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db9026afdbd458a8b14757279c92917)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.266794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:34.280135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83d5a243fd64e4e910800be64135b3a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.280979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83d5a243fd64e4e910800be64135b3a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.293819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83d5a243fd64e4e910800be64135b3a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.294812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83d5a243fd64e4e910800be64135b3a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.308655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9bf9ce8-0266-4121-8fb5-71b402749bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83d5a243fd64e4e910800be64135b3a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.329147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db9026afdbd458a8b14757279c92917'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.334537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 912/1808] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:34 2022] DELETE /identity/v3/users/b83d5a243fd64e4e910800be64135b3a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:34.338564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.338858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.339138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.343465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db9026afdbd458a8b14757279c92917'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.354459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db9026afdbd458a8b14757279c92917'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.355711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db9026afdbd458a8b14757279c92917'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.359121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:34.360334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:34.371023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d3e26667-2179-42b6-a4c6-bfc7b0d2f2ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db9026afdbd458a8b14757279c92917'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:34.379780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 897/1809] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:34 2022] DELETE /identity/v3/users/2db9026afdbd458a8b14757279c92917 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:34.384260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.384553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.385053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.405618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:34.406848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:34.432706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 913/1810] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:34.446144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.447667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:34.476267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 898/1811] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:34.486111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.487730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:34.503407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:34.514900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e7f5b-90f9-4423-b62f-d6e816e083ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.523144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:34.554705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f017a-af5f-4483-98ff-8a77e57b0817 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.559238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.560194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.560490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.560790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.567276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.567922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.568204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.568477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.570404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.571846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:34.578345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.579768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:34.582051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 899/1812] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:34.589991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 914/1813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:34.954036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:34.955516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:34.983956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ab8-b8d1-4d11-9387-1750cb3adc61 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:34.994785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:34.995585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:34.995967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:34.996345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] PATH_INFO: `/v3/projects/d2ccdd669c33405b8435b26d611dd182` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:34.997647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ccdd669c33405b8435b26d611dd182)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:34.999682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:35.055139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:35.056752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:35.066367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ccdd669c33405b8435b26d611dd182', 'request_id': 'req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.074474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ccdd669c33405b8435b26d611dd182', 'request_id': 'req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.080177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a8f45-fb23-4531-9364-dbf84a719809 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:35.080516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ccdd669c33405b8435b26d611dd182', 'request_id': 'req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.098818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] The token cache is being invalidate because project d2ccdd669c33405b8435b26d611dd182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:35.099177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.099917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.101625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 900/1814] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:34 2022] DELETE /identity/v3/projects/d2ccdd669c33405b8435b26d611dd182 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:35.105561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:35.106920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:35.167154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:35.167779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:35.168054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:35.168353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] PATH_INFO: `/v3/projects/90a16e5588df4f79905c64da5f3f5a7d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:35.169483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a16e5588df4f79905c64da5f3f5a7d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:35.171014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e30e10-9dcd-4c4b-b399-0b21bff8cf04 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:35.171382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:35.175980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:35.176639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:35.176944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:35.177249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] PATH_INFO: `/v3/users/c5f443ea312841f1928d271e5d2d50a4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:35.178476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f443ea312841f1928d271e5d2d50a4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:35.180557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:35.219373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a16e5588df4f79905c64da5f3f5a7d', 'request_id': 'req-d5a380cf-e77d-4a04-823c-3f93070146ca'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.223072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:35.228664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a16e5588df4f79905c64da5f3f5a7d', 'request_id': 'req-d5a380cf-e77d-4a04-823c-3f93070146ca'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.229515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f443ea312841f1928d271e5d2d50a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.234381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a16e5588df4f79905c64da5f3f5a7d', 'request_id': 'req-d5a380cf-e77d-4a04-823c-3f93070146ca'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.238677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f443ea312841f1928d271e5d2d50a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.246001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] The token cache is being invalidate because project 90a16e5588df4f79905c64da5f3f5a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:35.246289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.246479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f443ea312841f1928d271e5d2d50a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.246984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.247398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f443ea312841f1928d271e5d2d50a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.248464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 915/1815] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:35 2022] DELETE /identity/v3/projects/90a16e5588df4f79905c64da5f3f5a7d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:35.252273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:35.253889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:35.260839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f443ea312841f1928d271e5d2d50a4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.266774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 901/1816] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:35 2022] DELETE /identity/v3/users/c5f443ea312841f1928d271e5d2d50a4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:35.305875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:35.329833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a380cf-e77d-4a04-823c-3f93070146ca admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:35.392120 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:35.392768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:35.393052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:35.393362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] PATH_INFO: `/v3/users/25da6cbfd5d44cad88cf8c31610d3cf9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:35.394548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25da6cbfd5d44cad88cf8c31610d3cf9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:35.396511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:35.454094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25da6cbfd5d44cad88cf8c31610d3cf9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.454841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25da6cbfd5d44cad88cf8c31610d3cf9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.462379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25da6cbfd5d44cad88cf8c31610d3cf9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.463249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25da6cbfd5d44cad88cf8c31610d3cf9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.478159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f69eed86-eea8-4598-a67c-b46326701075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25da6cbfd5d44cad88cf8c31610d3cf9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.508374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 916/1817] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:35 2022] DELETE /identity/v3/users/25da6cbfd5d44cad88cf8c31610d3cf9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:35.601009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:35.603049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:35.618977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccb398-0f62-46f3-97f7-aec10a4a8338 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:35.762336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:35.763234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:35.763679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:35.765969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] PATH_INFO: `/v3/projects/260cfdfe5f43452ba4f53410c20f54f4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:35.767506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=260cfdfe5f43452ba4f53410c20f54f4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:35.770460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:35.840759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260cfdfe5f43452ba4f53410c20f54f4', 'request_id': 'req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.849498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260cfdfe5f43452ba4f53410c20f54f4', 'request_id': 'req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.857240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260cfdfe5f43452ba4f53410c20f54f4', 'request_id': 'req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.882422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] The token cache is being invalidate because project 260cfdfe5f43452ba4f53410c20f54f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:35.883052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.884123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02e79cd9-4fe3-4502-816c-4d79dcdcdc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:35.886343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 902/1818] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:35 2022] DELETE /identity/v3/projects/260cfdfe5f43452ba4f53410c20f54f4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:35.891731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:35.892024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:35.892904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:35.934366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:35.936088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.087060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 917/1819] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:36.088771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.089126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.089507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.173252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:36.178705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.199038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.200520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.227338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 903/1820] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:36.230418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed5df1-60c1-46dc-b5da-5dfa212adaa6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.231460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.243307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.243975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.244298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.244593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] PATH_INFO: `/v3/projects/85b61f8d81f144d28598ebba6b084b78` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.251183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b61f8d81f144d28598ebba6b084b78)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.253198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.255448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9ed52-e227-46e0-adbe-f43ae02bf97c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.267542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:36.269249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.270476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.270753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.271026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.271826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:36.273183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.279960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.314579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 904/1821] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:36.316746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.345346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc6b44-e8ec-4a68-83e9-6aa914ed14c8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.359167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.359818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.360275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.365036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.368010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.372672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.400883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b61f8d81f144d28598ebba6b084b78', 'request_id': 'req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.431690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b61f8d81f144d28598ebba6b084b78', 'request_id': 'req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.451146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b61f8d81f144d28598ebba6b084b78', 'request_id': 'req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.460641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 905/1822] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:36 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:36.474087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.492043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] The token cache is being invalidate because project 85b61f8d81f144d28598ebba6b084b78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:36.492524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.493338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fbab25c-a5e0-4bb7-b90f-adcb448e1b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.495731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 918/1823] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:36 2022] DELETE /identity/v3/projects/85b61f8d81f144d28598ebba6b084b78 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:36.500839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.502402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.502789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.512018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:36.607326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:36.607606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.689593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d21e8-db55-4cec-8db5-e9ab68345415 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.730671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 919/1824] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:36.732325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.737597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.738317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.738608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.738890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.739781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.740053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.741845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef427b-4016-4c68-bf39-9266d762363d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.792433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7555-b491-4eb6-a8e0-09e25d7b5399 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.797915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 906/1825] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:36 2022] POST /identity/v3/users => generated 294 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:36.797915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.797915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.797915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.806902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.808949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.810104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.811431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] PATH_INFO: `/v3/users/ecaea71e52ca46fca7dc57c50057f2c7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.818371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecaea71e52ca46fca7dc57c50057f2c7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.828475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.844347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:36.848968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.872244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 907/1826] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:36.873355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.876303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:36.899708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f47cd-e4a7-4d6f-9457-76926a77db42 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.907682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.908571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.909310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.909840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.910782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.913914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.943572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 908/1827] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:36.945001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:36.959231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d813780-6738-4a37-96cf-451b5d54559d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:36.967170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:36.967916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:36.968299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:36.969876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:36.969876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:36.970197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:36.974014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:36.986312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecaea71e52ca46fca7dc57c50057f2c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.989160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecaea71e52ca46fca7dc57c50057f2c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:36.999903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 909/1828] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:37.002140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.002640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecaea71e52ca46fca7dc57c50057f2c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:37.004061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecaea71e52ca46fca7dc57c50057f2c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:37.019281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d2f4c-78cd-4da6-80a4-4e0005e42499 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.019666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecaea71e52ca46fca7dc57c50057f2c7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:37.029601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.030348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.030874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.031226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.032018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.033869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.048085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 910/1829] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:37.049079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.051476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:37.069348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 920/1830] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:36 2022] DELETE /identity/v3/users/ecaea71e52ca46fca7dc57c50057f2c7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:37.071123 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.071229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f0d11-7a85-4fb6-a807-cdec520fce5e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.078043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:37.104214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:37.211343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e0296-0f82-4bc8-8b36-7ad5ccf6bde9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.217532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.217771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.218938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.219195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.219447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.220134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.220306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.220878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] PATH_INFO: `/v3/projects/6c0f61b19fa84116bebf00333191f59c/users/eb16ad25fb53472baeaa9e611dc64c38/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.221586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.224090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46721c-5547-4809-ba9e-fbfdf89e8cc5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.244827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0f61b19fa84116bebf00333191f59c, user_id=eb16ad25fb53472baeaa9e611dc64c38, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.248930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f4608-a48e-4907-9c74-246a939e3c8b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.272311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 921/1831] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:37 2022] PUT /identity/v3/projects/6c0f61b19fa84116bebf00333191f59c/users/eb16ad25fb53472baeaa9e611dc64c38/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:37.274633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.275426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.275919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.290269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 911/1832] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/projects => generated 367 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:37.291697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.292598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.292598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.325876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:37.328021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:37.332145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] MFA Rules not processed for user `eb16ad25fb53472baeaa9e611dc64c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:37.339109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:37.389584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:37.435358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 922/1833] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:37.436284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.460816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdaf0d-1d7d-4c1c-bb88-c104592198ed None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.472611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 912/1834] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:37.472611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.473803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:37.476587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.477587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.477918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.478581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.481074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.487077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.538906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 923/1835] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:37.544701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.565106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e4b64-33dd-449d-816e-129ba6101ae8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.573679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.574104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.574380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.574651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.575295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.577999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.632634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864a78d-0b0a-484d-b87a-71c6e2a66da8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.645679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.646545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.648759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.649217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.657819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 924/1836] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:37.662908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.677699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.680147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.724339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 913/1837] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:37.726904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.761753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98695ab-92d3-4bbd-8afa-72cbc6c0fa05 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.788403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:37.844131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830bf9f-70bf-4acf-90b4-b696ec5d156c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.862007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.864051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.864344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.865565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.866305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.870858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.904904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 925/1838] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:37.912018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.925000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a04e35-a5f1-40d8-ae80-c7ec4b580f7f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:37.932313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.933446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.940894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.941266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.943085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.945190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:37.970185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:37.970862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:37.971906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:37.972193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:37.977488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 926/1839] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:37.978793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:37.979553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:37.984637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.014101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54bf3e-bce9-486a-ad21-e9c4352f4de1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.024664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.026177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.026516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.026802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] PATH_INFO: `/v3/projects/529dc955e99d479ab854ef54198875b6/users/bbbb13cb210a421e8eaec410effb15fb/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.048705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 914/1840] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:37 2022] POST /identity/v3/users => generated 299 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:38.051301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.075461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=529dc955e99d479ab854ef54198875b6, user_id=bbbb13cb210a421e8eaec410effb15fb, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.077011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae9435-6774-421e-8b0f-47130903c96d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.080023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a331e5-65fe-4bdb-a121-5ff20aba6ec9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.088144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.088685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.088970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.089301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.091290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.098586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.134967 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 927/1841] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:37 2022] PUT /identity/v3/projects/529dc955e99d479ab854ef54198875b6/users/bbbb13cb210a421e8eaec410effb15fb/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:38.139709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.140297 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.141460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.161595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 915/1842] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:38.166405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.211829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84bbdb-3e1c-454d-87aa-1150fcb1fad9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.292825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] MFA Rules not processed for user `bbbb13cb210a421e8eaec410effb15fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:38.298054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:38.405727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:38.434686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.435380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.435656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.436100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] PATH_INFO: `/v3/projects/d082bd4478c84a83844845f1207e4e6e/users/79a9ce69feed475c8087ec1fab13648d/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.495151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d082bd4478c84a83844845f1207e4e6e, user_id=79a9ce69feed475c8087ec1fab13648d, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.498188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.512871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 928/1843] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:38.518398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.538672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 916/1844] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:38 2022] PUT /identity/v3/projects/d082bd4478c84a83844845f1207e4e6e/users/79a9ce69feed475c8087ec1fab13648d/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:38.542815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.544306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04786a6-1656-4096-a4e8-479dfaf0663c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.640147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.641067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.642343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.642775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.643575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.645762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.668864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ba961-f721-4126-bf3c-b3bb7b08aa2f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.674907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.675605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.675972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.676816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.677570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.680331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.700911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 929/1845] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:38 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:38.702600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.705719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:38.710564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 917/1846] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:38.711716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.725514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf3b9-8a85-4438-a9a6-5a113543003b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.774031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:38.829842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c314cb-78a6-470b-998d-741fab75259c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.835292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.835941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.836198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.836476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.837398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.841901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.844922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.845609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.845891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.846761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] PATH_INFO: `/v3/projects/b58647e15fa343b8b75bf51e7f0792ab` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.847729 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58647e15fa343b8b75bf51e7f0792ab)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.850288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.878487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 918/1847] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:38 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:38.879372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.893172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4db16-86fd-4e90-ba0d-d2483c284252 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:38.898702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:38.899495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.899861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.900209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] PATH_INFO: `/v3/projects/d082bd4478c84a83844845f1207e4e6e/users/79a9ce69feed475c8087ec1fab13648d/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:38.902617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d082bd4478c84a83844845f1207e4e6e, user_id=79a9ce69feed475c8087ec1fab13648d, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:38.906642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:38.920715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 919/1848] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:38 2022] PUT /identity/v3/projects/d082bd4478c84a83844845f1207e4e6e/users/79a9ce69feed475c8087ec1fab13648d/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:38.921509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:38.944414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58647e15fa343b8b75bf51e7f0792ab', 'request_id': 'req-085fecec-5a73-4133-9a77-d20ebb6ec90e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:38.953326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58647e15fa343b8b75bf51e7f0792ab', 'request_id': 'req-085fecec-5a73-4133-9a77-d20ebb6ec90e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:38.962668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58647e15fa343b8b75bf51e7f0792ab', 'request_id': 'req-085fecec-5a73-4133-9a77-d20ebb6ec90e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:38.981040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:38.984654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] The token cache is being invalidate because project b58647e15fa343b8b75bf51e7f0792ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:38.985248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:38.987137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-085fecec-5a73-4133-9a77-d20ebb6ec90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:38.996286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 930/1849] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:38 2022] DELETE /identity/v3/projects/b58647e15fa343b8b75bf51e7f0792ab => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:38.997891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:38.998177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:38.998502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.014649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e9f7b-fa46-4f7f-9dfa-b5be6e0f3bea admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.033235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] MFA Rules not processed for user `79a9ce69feed475c8087ec1fab13648d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:39.035222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.090627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.091254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.091572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.091802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.092452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.094218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.112213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 920/1850] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:39.114504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.117064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.119493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 931/1851] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:39.122207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.123966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.134763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3805e0f-ad50-4861-921b-aa4b3373f336 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.139366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa824d-cf6e-4ab7-9591-c21dda143619 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.139722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.140356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.140630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.140899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] PATH_INFO: `/v3/users/5f214fd3010c4e1c96ea6a281bf42524` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.142047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f214fd3010c4e1c96ea6a281bf42524)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.144094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.144307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.145000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.145312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.145590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] PATH_INFO: `/v3/projects/753e732f544e442da8be075358f451a6/users/d2f1cbb803a145479f11497950540509/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.164138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] RBAC: Authorizing `identity:create_grant(project_id=753e732f544e442da8be075358f451a6, user_id=d2f1cbb803a145479f11497950540509, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.166966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.182993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 932/1852] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:39 2022] PUT /identity/v3/projects/753e732f544e442da8be075358f451a6/users/d2f1cbb803a145479f11497950540509/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:39.186321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.186952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.195272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f214fd3010c4e1c96ea6a281bf42524'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.203689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f214fd3010c4e1c96ea6a281bf42524'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.212571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:39.239801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f214fd3010c4e1c96ea6a281bf42524'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.240851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f214fd3010c4e1c96ea6a281bf42524'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.268154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d187b1-3ccd-47f1-bd44-48dad61d647e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.270589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f214fd3010c4e1c96ea6a281bf42524'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.272887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.273526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.273810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.274087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.275845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.276755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 921/1853] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:39 2022] DELETE /identity/v3/users/5f214fd3010c4e1c96ea6a281bf42524 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:39.279880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.281357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.369189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.370831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.378605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880208b-7296-4dd8-ad33-4dd60e8d50c7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.383102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 933/1854] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:39.393623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.394960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.395371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.395815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.396553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.398434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e9dda-fa55-4090-b0cf-26f3fd1705ea admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.405790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:39.415587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 922/1855] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:39.416774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.507900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a1c4-74b3-452c-a7b3-668c6ed2ae39 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.513945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.514620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.514906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.515189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] PATH_INFO: `/v3/projects/753e732f544e442da8be075358f451a6/users/d2f1cbb803a145479f11497950540509/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.518017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=753e732f544e442da8be075358f451a6, user_id=d2f1cbb803a145479f11497950540509, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.521217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.534209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 934/1856] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:39 2022] PUT /identity/v3/projects/753e732f544e442da8be075358f451a6/users/d2f1cbb803a145479f11497950540509/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:39.545897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.546189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.546545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.576443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] MFA Rules not processed for user `d2f1cbb803a145479f11497950540509`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:39.577756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.655558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.657518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.677651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 923/1857] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:39.699113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.700639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.745890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f4fdc-6755-43b6-a5b0-8cb1960548dd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.751535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.753199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.753199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.753199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] PATH_INFO: `/v3/projects/d48530e5d0a1470aa2ee149581710114` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.754329 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48530e5d0a1470aa2ee149581710114)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.757240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.760266 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:39.777543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d4142-60c4-42ad-852e-e26eebac480d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.782449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.783081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.783362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.783656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.796128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:39.798039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5631c-6b37-48a7-922c-0165b8c346f4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:39.809793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 924/1858] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:39.819646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48530e5d0a1470aa2ee149581710114', 'request_id': 'req-e1dd9142-8401-4c5e-92bf-ec13950b8afb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.833621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48530e5d0a1470aa2ee149581710114', 'request_id': 'req-e1dd9142-8401-4c5e-92bf-ec13950b8afb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.839735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48530e5d0a1470aa2ee149581710114', 'request_id': 'req-e1dd9142-8401-4c5e-92bf-ec13950b8afb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.852250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] The token cache is being invalidate because project d48530e5d0a1470aa2ee149581710114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:39.852550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.853301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:39.854825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 935/1859] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:39 2022] DELETE /identity/v3/projects/d48530e5d0a1470aa2ee149581710114 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:39.859529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0def671-ccd2-459b-a804-73a5a7983270 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.860092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0def671-ccd2-459b-a804-73a5a7983270 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.860563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0def671-ccd2-459b-a804-73a5a7983270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.860950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:39.862593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.883409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e0def671-ccd2-459b-a804-73a5a7983270 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:39.884713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0def671-ccd2-459b-a804-73a5a7983270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.943475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd9142-8401-4c5e-92bf-ec13950b8afb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:39.947778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:39.948406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.949283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.949283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.950756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:39.966476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 925/1860] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:39.975556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:39.975900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:39.976289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:39.997341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:39.998695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.007673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 926/1861] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:40.013436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.024278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf03b-249b-4aff-b228-b3390369cda6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.030568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.031018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.031716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.032061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.032373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.032679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.033530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:40.033902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.036368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.044627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 936/1862] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:40.057287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 927/1863] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:40.060124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.088577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0481b2-3c94-46d6-8f42-dd71e417e16a service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.098790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.099475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.099764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.100041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.100689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.102904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.146818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 937/1864] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/projects => generated 399 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:40.148229 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.159456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4749-72a8-4a79-9b3b-9e4f25df903e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.165947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:40.199062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.200961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.251898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.252499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.252796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.253058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.253900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.256983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.259745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:40.287515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 928/1865] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/users => generated 300 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:40.291635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.306310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20be032-3df9-4ca8-a8c1-eed2746378ae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.328178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.330839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.333186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.333902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.335688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.338544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.356382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557c0d9-ad6f-4cd8-872c-1691e4548ba3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.364476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.365163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.365444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.365722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] PATH_INFO: `/v3/users/eb16ad25fb53472baeaa9e611dc64c38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.371082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 929/1866] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:40.371211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb16ad25fb53472baeaa9e611dc64c38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.371649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.372630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.394144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420fb58-5107-4e4f-9688-c2af9d4b2cf9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.406815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.407477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.408537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.408824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.409512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.411293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.423476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 930/1867] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:40.427379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.441332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cef69-0244-4aac-81e9-95f81aa74d46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.447077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.447677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.447955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.448234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] PATH_INFO: `/v3/projects/f6a2f09dd1df44afb5f2d20486bf20cf/users/3152c36daf6e4de291bbcfa1c52222e0/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.466694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a2f09dd1df44afb5f2d20486bf20cf, user_id=3152c36daf6e4de291bbcfa1c52222e0, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.469659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90351-7c67-45f0-a225-59fc69460df8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.487048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 931/1868] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:40 2022] PUT /identity/v3/projects/f6a2f09dd1df44afb5f2d20486bf20cf/users/3152c36daf6e4de291bbcfa1c52222e0/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:40.498810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb16ad25fb53472baeaa9e611dc64c38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:40.499526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb16ad25fb53472baeaa9e611dc64c38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:40.502023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.502176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.502782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.511522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb16ad25fb53472baeaa9e611dc64c38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:40.512742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb16ad25fb53472baeaa9e611dc64c38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:40.536435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-042795f8-a75f-4160-b025-d394f7843897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb16ad25fb53472baeaa9e611dc64c38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:40.545677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] MFA Rules not processed for user `3152c36daf6e4de291bbcfa1c52222e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:40.547718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.558952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 938/1869] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:40 2022] DELETE /identity/v3/users/eb16ad25fb53472baeaa9e611dc64c38 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:40.563732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.564159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.564446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.585801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:40.587016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.624105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 932/1870] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:40.628692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.630612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.649985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 939/1871] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:40.660043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.661449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:40.678269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:40.701878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72647c7-2ae7-4ba7-8bc1-f6d0bd31fa9f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.706478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.707084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.707435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.707742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.708395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.710203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:40.710391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.721396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93debe30-2c35-4d5f-a6b1-78d3341baf42 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.725683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.726325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.726626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.726900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.736369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.737867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.742474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 933/1872] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:40.743432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.748488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 940/1873] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:40.813675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06cbe5-6951-4936-993c-410606054bb9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.819448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.820230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.820505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.820777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.821474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.823436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.851108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 934/1874] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:40 2022] POST /identity/v3/users => generated 306 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:40.852155 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.861758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4a577-4a65-4ab7-8525-ebf6ecff8cb3 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.867163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.867760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.868040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.868353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.869016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.870623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.881717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 941/1875] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:40.886049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.896470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916751f-109d-411f-8fd9-be6b7df4b93e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:40.901695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:40.902819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:40.903099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:40.903377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] PATH_INFO: `/v3/projects/6f43069e37d446b18fa2ff56370b0868/users/c2dcf70ad88d49d684ad08fabea29394/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:40.919688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f43069e37d446b18fa2ff56370b0868, user_id=c2dcf70ad88d49d684ad08fabea29394, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:40.922476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:40.937648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 935/1876] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:40 2022] PUT /identity/v3/projects/6f43069e37d446b18fa2ff56370b0868/users/c2dcf70ad88d49d684ad08fabea29394/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:40.942247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:40.953689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f80d3f-4fdd-46b9-85b3-a5a2b04594de admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.014979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.015401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:41.015600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.015882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.016234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.016806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.017347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:41.018454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.030782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 942/1877] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:41.034710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:41.034897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac304aa-4823-4587-b367-6af977f4ed71 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.041241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.041918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.043074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.043074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] PATH_INFO: `/v3/projects/6c0f61b19fa84116bebf00333191f59c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.044001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0f61b19fa84116bebf00333191f59c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.046000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.047105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0538a-d667-4a85-a97e-49a9724dec7c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.052816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.053383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.053664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.053942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] PATH_INFO: `/v3/projects/6f43069e37d446b18fa2ff56370b0868/users/c2dcf70ad88d49d684ad08fabea29394/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.055897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f43069e37d446b18fa2ff56370b0868, user_id=c2dcf70ad88d49d684ad08fabea29394, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.058573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6b1ff-12ce-4026-947a-6b9fe67c428f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.070351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 943/1878] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:41 2022] PUT /identity/v3/projects/6f43069e37d446b18fa2ff56370b0868/users/c2dcf70ad88d49d684ad08fabea29394/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:41.082462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.082755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.083033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.102323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0f61b19fa84116bebf00333191f59c', 'request_id': 'req-8eb13048-7dd5-4c44-b5f2-7a253539600b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.108550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0f61b19fa84116bebf00333191f59c', 'request_id': 'req-8eb13048-7dd5-4c44-b5f2-7a253539600b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.111580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] MFA Rules not processed for user `c2dcf70ad88d49d684ad08fabea29394`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:41.112865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:41.114724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0f61b19fa84116bebf00333191f59c', 'request_id': 'req-8eb13048-7dd5-4c44-b5f2-7a253539600b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.127857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] The token cache is being invalidate because project 6c0f61b19fa84116bebf00333191f59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:41.128146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.128798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.130487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 936/1879] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:41 2022] DELETE /identity/v3/projects/6c0f61b19fa84116bebf00333191f59c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:41.134479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:41.135782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:41.148843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:41.182643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 944/1880] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:41.194377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:41.196402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:41.206976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb13048-7dd5-4c44-b5f2-7a253539600b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.211912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.212555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.212837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.213134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] PATH_INFO: `/v3/users/79a9ce69feed475c8087ec1fab13648d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.214323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a9ce69feed475c8087ec1fab13648d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.216258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.250806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:41.263158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d632-cd1f-45ec-aff8-5ffeeda3c9e0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.266821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.267455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.267566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a9ce69feed475c8087ec1fab13648d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.267736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.268012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.274378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a9ce69feed475c8087ec1fab13648d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.277020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.278484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.281599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a9ce69feed475c8087ec1fab13648d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.282254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a9ce69feed475c8087ec1fab13648d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.288717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 945/1881] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:41.298605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a9ce69feed475c8087ec1fab13648d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:41.305265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 937/1882] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:41 2022] DELETE /identity/v3/users/79a9ce69feed475c8087ec1fab13648d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:41.383471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:41.393903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d4ded-0d05-4df0-8709-b80bfc9e7203 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:41.449935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:41.450985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:41.450985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:41.450985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:41.452866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:41.529487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:41.530964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a8344-bd87-4da3-80f4-0e63a0e6689a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:41.541608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 946/1883] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:42.500738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:42.502287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:42.585952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8fc49-d761-4f43-8006-28e768eec13d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:42.593019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:42.593710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:42.594045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:42.594323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] PATH_INFO: `/v3/projects/d082bd4478c84a83844845f1207e4e6e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:42.595489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d082bd4478c84a83844845f1207e4e6e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:42.597455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:42.646730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:42.662455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd082bd4478c84a83844845f1207e4e6e', 'request_id': 'req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:42.670185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd082bd4478c84a83844845f1207e4e6e', 'request_id': 'req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:42.681786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd082bd4478c84a83844845f1207e4e6e', 'request_id': 'req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:42.703983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] The token cache is being invalidate because project d082bd4478c84a83844845f1207e4e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:42.704333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:42.705178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fbeafdc6-8cf2-4b50-9c6b-b55a1534098a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:42.706816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 938/1884] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:42 2022] DELETE /identity/v3/projects/d082bd4478c84a83844845f1207e4e6e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:42.713449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:42.715148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:42.715621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:42.742109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:42.743415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:42.851038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 947/1885] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:42.858790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:42.859077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:42.859353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:42.882473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:42.884379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:42.897791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 939/1886] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:42.899796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:42.917903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e90e3-d78f-4042-975a-72662beca1e8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:42.924884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:42.925506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:42.926489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:42.927332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:42.928241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:42.929981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:42.932510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:42.965839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 948/1887] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:42.969619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:42.981687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5be28-e4ee-40bc-a52d-d88c0665bb81 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:42.986738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:42.987604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:42.987892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:42.989281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:42.989925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:42.992535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.024270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 940/1888] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:42 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:43.028160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.053595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab080ff-a762-4d1c-9b01-4098e8531f84 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.085501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:43.118629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.119236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.119512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.119787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.120426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.122365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.144597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 949/1889] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:43 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:43.148776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.159496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5645-8748-4aad-89b1-d2b08aac9d6f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.166323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.166921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.167200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.167470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.168105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.169754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.180473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 941/1890] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:43.184442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.195754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac733a2-1f90-4790-844b-5d934528beb3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.200877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.201655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.201938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.202221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.202866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.204463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf37b3-620f-4829-a60d-3654bc4086f2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.216033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 950/1891] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:43.220148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.231742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6d8d4-bd92-48ca-9dad-8eac1551c4aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.236579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.237512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.237854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.238138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] PATH_INFO: `/v3/projects/9513ec5b3c32485dbf088369ab1bd091/users/3c3dac53378a4948b6bd9c717b62decb/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.256250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9513ec5b3c32485dbf088369ab1bd091, user_id=3c3dac53378a4948b6bd9c717b62decb, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.259183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.277222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 942/1892] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:43 2022] PUT /identity/v3/projects/9513ec5b3c32485dbf088369ab1bd091/users/3c3dac53378a4948b6bd9c717b62decb/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:43.288871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.289189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.289547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.319932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.321690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.324069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] MFA Rules not processed for user `3c3dac53378a4948b6bd9c717b62decb`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:43.325579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.475267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 951/1893] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:43.481326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46505d57-fb96-4600-8dd3-67b57771a97d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.498676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.500650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.501342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.501940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.502122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.502122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] PATH_INFO: `/v3/users/b461f8c619694b89ad36f3b71b5b434e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.503414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b461f8c619694b89ad36f3b71b5b434e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.508184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.584564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d4e2-968c-40ea-9dd5-53836463f9d8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.589802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.590597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.590888 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.591197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.598927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b461f8c619694b89ad36f3b71b5b434e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.602610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.604608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.616450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 952/1894] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:43.616691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b461f8c619694b89ad36f3b71b5b434e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.622236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:43.631567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b461f8c619694b89ad36f3b71b5b434e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.632986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b461f8c619694b89ad36f3b71b5b434e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.640725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.650690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-23e7bf4a-5881-4ad8-b1b0-c73bae62408a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b461f8c619694b89ad36f3b71b5b434e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.654562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20d579-ef62-47a5-9802-4551da7d8f60 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.659264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 943/1895] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:43 2022] DELETE /identity/v3/users/b461f8c619694b89ad36f3b71b5b434e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:43.664663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.665214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.665495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.665782 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] PATH_INFO: `/v3/users/3c3dac53378a4948b6bd9c717b62decb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.666867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.667058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3dac53378a4948b6bd9c717b62decb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:43.667447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.667866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.668984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:43.707866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:43.709833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.733283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:43.751508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3dac53378a4948b6bd9c717b62decb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.757044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3dac53378a4948b6bd9c717b62decb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.773718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3dac53378a4948b6bd9c717b62decb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.776079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3dac53378a4948b6bd9c717b62decb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.788622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d247468-82a0-40de-98fe-b28ec73660b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3dac53378a4948b6bd9c717b62decb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:43.806902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 944/1896] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:43.816569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.818533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.825207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 953/1897] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:43 2022] DELETE /identity/v3/users/3c3dac53378a4948b6bd9c717b62decb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:43.826634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.826922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.827205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:43.851668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:43.852924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:43.919589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaf5f4-b0a5-4ebd-8769-1b40019be68b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:43.947710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 954/1898] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:43.959152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:43.986998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:43.987699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:43.988046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:43.989000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.000180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.002169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.018003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 945/1899] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:44.062691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d5135-6788-4f30-a182-9ddb7bff0abc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:44.068394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:44.070418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.071064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.071605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.086151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.088458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a185a3b-7cd4-4cc1-97b8-ad9d7176c5e8 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.115928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 955/1900] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:44.145025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:44.147511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.165491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942785f-f6b9-4edc-8605-cf0576059abf service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:44.170143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:44.170896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.171273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.171642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] PATH_INFO: `/v3/users/bbbb13cb210a421e8eaec410effb15fb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.173428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbb13cb210a421e8eaec410effb15fb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.175703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.258439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:44.289552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbb13cb210a421e8eaec410effb15fb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.311135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbb13cb210a421e8eaec410effb15fb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.327119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbb13cb210a421e8eaec410effb15fb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.328372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbb13cb210a421e8eaec410effb15fb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.353934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbb13cb210a421e8eaec410effb15fb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.363002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 946/1901] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:44 2022] DELETE /identity/v3/users/bbbb13cb210a421e8eaec410effb15fb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:44.367777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.368087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.368365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.405827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:44.407909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.515443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:44.517208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.547469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:44.603299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 956/1902] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:44.649755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:44.651474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.694906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba42c67-8fc2-48d0-be81-07aad0cfa36b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:44.712484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:44.713195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.713515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.713779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] PATH_INFO: `/v3/projects/9f10e8c5e7694446a81ea085a367342f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.716755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f10e8c5e7694446a81ea085a367342f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.718717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.772673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd4116-bcf0-482c-8a8f-554e6cce60aa None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:44.779028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:44.779643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.779918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.780303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.791014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.792535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.805494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 957/1903] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:44.806469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:44.807673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.822234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaa4a8-2dbd-49d8-82be-c32c9247db15 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:44.830898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:44.831545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:44.831895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:44.832193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] PATH_INFO: `/v3/projects/9513ec5b3c32485dbf088369ab1bd091` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:44.833607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9513ec5b3c32485dbf088369ab1bd091)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:44.838270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:44.878117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f10e8c5e7694446a81ea085a367342f', 'request_id': 'req-cec09cbc-72ff-4f91-a5d4-d68344203a88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.888864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f10e8c5e7694446a81ea085a367342f', 'request_id': 'req-cec09cbc-72ff-4f91-a5d4-d68344203a88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.897397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f10e8c5e7694446a81ea085a367342f', 'request_id': 'req-cec09cbc-72ff-4f91-a5d4-d68344203a88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.922896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] The token cache is being invalidate because project 9f10e8c5e7694446a81ea085a367342f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:44.923212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.927824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.927824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 947/1904] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:44 2022] DELETE /identity/v3/projects/9f10e8c5e7694446a81ea085a367342f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:44.930256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:44.932232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:44.942056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9513ec5b3c32485dbf088369ab1bd091', 'request_id': 'req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.952289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9513ec5b3c32485dbf088369ab1bd091', 'request_id': 'req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.960571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9513ec5b3c32485dbf088369ab1bd091', 'request_id': 'req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.981314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] The token cache is being invalidate because project 9513ec5b3c32485dbf088369ab1bd091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:44.981629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.982767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ba80bb5d-6f9f-41e4-8712-ee7da3f6433f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:44.998693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:45.001600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 958/1905] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:44 2022] DELETE /identity/v3/projects/9513ec5b3c32485dbf088369ab1bd091 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:45.004817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c38b6b-a497-4be1-8dfe-52242c6a8b67 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.005105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c38b6b-a497-4be1-8dfe-52242c6a8b67 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.005395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c38b6b-a497-4be1-8dfe-52242c6a8b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.020518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec09cbc-72ff-4f91-a5d4-d68344203a88 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.037697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-41c38b6b-a497-4be1-8dfe-52242c6a8b67 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:45.038975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41c38b6b-a497-4be1-8dfe-52242c6a8b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.094088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.094923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.095208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.095553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] PATH_INFO: `/v3/users/e81f3a96b1c84c1b8428888d205ca0a6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.098130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81f3a96b1c84c1b8428888d205ca0a6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.100559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.113264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 959/1906] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:45.121241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.121919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.122466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.152892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:45.154475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.163088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 960/1907] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:45.167818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.169706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81f3a96b1c84c1b8428888d205ca0a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.177254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95a54a-77f2-4b5e-9857-55765c2c2b27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.177884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81f3a96b1c84c1b8428888d205ca0a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.183109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.184177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.184292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.184498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.185304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:45.185613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.187641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.190099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81f3a96b1c84c1b8428888d205ca0a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.192691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81f3a96b1c84c1b8428888d205ca0a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.204488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 961/1908] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:14:45.208869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.211170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.213660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81f3a96b1c84c1b8428888d205ca0a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.220653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 948/1909] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:44 2022] DELETE /identity/v3/users/e81f3a96b1c84c1b8428888d205ca0a6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:45.226785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:45.236291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3c82a-511d-495f-9a44-4e7c339ef4f3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.348880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.349617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.350058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.350345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.351831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.354526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.397801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 962/1910] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:45 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:45.398681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.413512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3be4f8-f5bd-4826-96ab-28ae00c9a1a2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.475962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.477772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.564919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.565709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.565994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.566291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.567183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.568830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.582163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:45.593832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b9ad-fbb8-494a-a777-db1a59071d84 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.595133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 949/1911] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:45 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:45.600070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.603018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.603766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.604086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.604430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] PATH_INFO: `/v3/projects/529dc955e99d479ab854ef54198875b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.605657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=529dc955e99d479ab854ef54198875b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.607930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.615206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c6b17-862b-42b6-a934-b8abeac74bf1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.621475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.622166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.622513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.622856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.623614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.625541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.640477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 950/1912] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:45.641719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.653222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962923a-a855-45b7-be2b-faf61b66471a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.658971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.659579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.659859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.660133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.661361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.663123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.677608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529dc955e99d479ab854ef54198875b6', 'request_id': 'req-1ed533c9-da93-493d-888d-c2016c2550b7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.680172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 951/1913] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:45.684420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.695923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881aab34-e365-4f36-8c2b-8358ddd95518 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.708841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529dc955e99d479ab854ef54198875b6', 'request_id': 'req-1ed533c9-da93-493d-888d-c2016c2550b7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.728535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529dc955e99d479ab854ef54198875b6', 'request_id': 'req-1ed533c9-da93-493d-888d-c2016c2550b7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.742566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] The token cache is being invalidate because project 529dc955e99d479ab854ef54198875b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:45.742852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.743586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.745151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 963/1914] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:45 2022] DELETE /identity/v3/projects/529dc955e99d479ab854ef54198875b6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:45.758019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.759651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.789948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:45.824963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.826205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.827184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.827543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] PATH_INFO: `/v3/projects/0527e01480e44ca4b314cbca75204a0f/users/3b845fd64b904d71a41e054c3a2330d5/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.845435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0527e01480e44ca4b314cbca75204a0f, user_id=3b845fd64b904d71a41e054c3a2330d5, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.849072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.851374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed533c9-da93-493d-888d-c2016c2550b7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:45.858876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:45.859630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.859975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.860244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] PATH_INFO: `/v3/projects/795843e364604efd8d73875152ea8ffc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.861458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] RBAC: Authorizing `identity:delete_project(project_id=795843e364604efd8d73875152ea8ffc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:45.863828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:45.878323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 952/1915] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:45 2022] PUT /identity/v3/projects/0527e01480e44ca4b314cbca75204a0f/users/3b845fd64b904d71a41e054c3a2330d5/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:45.878323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:45.878323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:45.944649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795843e364604efd8d73875152ea8ffc', 'request_id': 'req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.961282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795843e364604efd8d73875152ea8ffc', 'request_id': 'req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.968496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795843e364604efd8d73875152ea8ffc', 'request_id': 'req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.984617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] The token cache is being invalidate because project 795843e364604efd8d73875152ea8ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:45.985334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.986326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a0c460b6-b6db-4ec1-a7e0-69cfc9ad962f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:45.988546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 964/1916] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:45 2022] DELETE /identity/v3/projects/795843e364604efd8d73875152ea8ffc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:45.990181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2dc08-ca91-41c3-a4b0-6f50ba4ea53f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:45.990780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2dc08-ca91-41c3-a4b0-6f50ba4ea53f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:45.991190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2dc08-ca91-41c3-a4b0-6f50ba4ea53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:45.993342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc59e72-39aa-4707-b131-f92e169248a0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.054564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-08b2dc08-ca91-41c3-a4b0-6f50ba4ea53f None None] MFA Rules not processed for user `3b845fd64b904d71a41e054c3a2330d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:46.056484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-08b2dc08-ca91-41c3-a4b0-6f50ba4ea53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.187371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 965/1917] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:46.188696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.189028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.189345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.198394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.200310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.201432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.201760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] PATH_INFO: `/v3/users/d2f1cbb803a145479f11497950540509` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.204498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f1cbb803a145479f11497950540509)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.210352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.220013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:46.225667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:46.227004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.242528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 966/1918] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:46.243492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.245023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.293200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:46.295649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f1cbb803a145479f11497950540509'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.307619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f1cbb803a145479f11497950540509'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.317921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f1cbb803a145479f11497950540509'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.318551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f1cbb803a145479f11497950540509'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.333284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8aefd11-f849-411a-aef7-947c802b891e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f1cbb803a145479f11497950540509'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.338381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 953/1919] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:45 2022] DELETE /identity/v3/users/d2f1cbb803a145479f11497950540509 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:46.341377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.341377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.341377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.358029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ab61c-bf7d-43ec-9cec-c167c1805589 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.361776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:46.363010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.415195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.415918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.416177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.416454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.418627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.439308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 954/1920] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:46.440383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.442385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.455628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab9a33-9e79-47ed-bd1e-825af90d295c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.461334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.462084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.462285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.462531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] PATH_INFO: `/v3/users/3152c36daf6e4de291bbcfa1c52222e0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.463682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3152c36daf6e4de291bbcfa1c52222e0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.465710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.490937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.492776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.504168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 967/1921] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:46.505439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.518114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb7002-7350-4b08-9e3e-f7da2ad5fc60 service glance] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.523457 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.524139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.527674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.527813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3152c36daf6e4de291bbcfa1c52222e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.527910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.527910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:46.527910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.527910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.537216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3152c36daf6e4de291bbcfa1c52222e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.541192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 968/1922] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:46.546456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.547493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3152c36daf6e4de291bbcfa1c52222e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.548521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3152c36daf6e4de291bbcfa1c52222e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.563435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:46.565228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e36af63a-c906-488c-bb96-5b53bf054013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3152c36daf6e4de291bbcfa1c52222e0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.570958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897f5ef-840c-4829-b185-79e4bd92a999 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.575147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 955/1923] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:46 2022] DELETE /identity/v3/users/3152c36daf6e4de291bbcfa1c52222e0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:46.579789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.580155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.580263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.580591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.581759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.582049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.582915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.584300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.586923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.607361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:46.610062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.623259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 969/1924] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:46.624959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.629291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.702888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 956/1925] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:46.704137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.705768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.717911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:46.720105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513fb1-0c4c-4b6a-abb7-e3b444677453 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.739455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319b6ab-cfaf-41b7-bcbc-a46d541b70c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.744684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.745676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.746285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.746615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.747336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.749704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.791284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 970/1926] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:46.791284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.792835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:46.841794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.842771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.843036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.843313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] PATH_INFO: `/v3/projects/753e732f544e442da8be075358f451a6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.845197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] RBAC: Authorizing `identity:delete_project(project_id=753e732f544e442da8be075358f451a6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.849161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.878884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b337b3-c015-4907-aa7d-006d4ea340f3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.884528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:46.885190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.885476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.885745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:46.900315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:46.902016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:46.915487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 971/1927] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:46.923759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:46.926208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753e732f544e442da8be075358f451a6', 'request_id': 'req-a3614e91-981f-4f15-94c2-4e8ff33fa68c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.937332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753e732f544e442da8be075358f451a6', 'request_id': 'req-a3614e91-981f-4f15-94c2-4e8ff33fa68c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.947082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb5c15-c776-46be-b1e5-f4fb20efea6a service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:46.947638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753e732f544e442da8be075358f451a6', 'request_id': 'req-a3614e91-981f-4f15-94c2-4e8ff33fa68c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.971131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] The token cache is being invalidate because project 753e732f544e442da8be075358f451a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:46.971518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.972398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3614e91-981f-4f15-94c2-4e8ff33fa68c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:46.974945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 957/1928] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:46 2022] DELETE /identity/v3/projects/753e732f544e442da8be075358f451a6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:46.979610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09054fbe-eff4-4ac8-a416-0dd971bb3d39 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:46.979957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09054fbe-eff4-4ac8-a416-0dd971bb3d39 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:46.980341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09054fbe-eff4-4ac8-a416-0dd971bb3d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.006807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-09054fbe-eff4-4ac8-a416-0dd971bb3d39 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:47.008391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09054fbe-eff4-4ac8-a416-0dd971bb3d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:47.068766 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.069347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.069603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.069796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:47.069946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.070603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.072253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.094908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 972/1929] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:47.100283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.106527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:47.133877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85415a-a764-4878-b869-e6d3a93ea981 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.141328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.142119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.142560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.142989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.144322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.146691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.171890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 973/1930] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:47.181835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.186517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 958/1931] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:47.195943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.196236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.197382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.220804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48e0a-742e-4d50-9f87-174f38f300e4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.235861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.236804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.239576 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.240172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] PATH_INFO: `/v3/projects/cc6ce0d5670640368fd4b1745f3c543c/users/b18d8d413b15494ab4369490bf0ff9ad/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.243502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:47.250441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:47.281316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6ce0d5670640368fd4b1745f3c543c, user_id=b18d8d413b15494ab4369490bf0ff9ad, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.281715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 959/1932] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:47.281861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c30f1-29fa-4948-891e-01d662a47da3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.282134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.294708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14065ce-80a4-4cdf-834d-7441b3313df5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.301032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.302406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.302794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.303148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.304361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:47.304768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.306840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.325360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 974/1933] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:47 2022] PUT /identity/v3/projects/cc6ce0d5670640368fd4b1745f3c543c/users/b18d8d413b15494ab4369490bf0ff9ad/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:47.331120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 960/1934] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:47.331693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.331989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.332837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.336362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.357910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8095645-0d72-4d46-99b2-3e24a23b11c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.400157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] MFA Rules not processed for user `b18d8d413b15494ab4369490bf0ff9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:47.401467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:47.448410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:47.501717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.503454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.505263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.507278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.508742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.513372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.539276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 975/1935] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:47.548160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.594586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b1ea3-6068-4fdb-9ea5-2dc31bf1a636 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.603772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 961/1936] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/projects => generated 385 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:47.604749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.675775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.678691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.679165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.679570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.680412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.683412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.756596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 976/1937] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:47.756596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.763298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:47.787902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b00e9-1195-479d-a0ea-288d48026f7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.833542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:47.848427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dc44e-07e2-4dd6-999d-051ae3ebbf07 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.854633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.855336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.855539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.855812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] PATH_INFO: `/v3/projects/f6a2f09dd1df44afb5f2d20486bf20cf` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.857303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a2f09dd1df44afb5f2d20486bf20cf)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.859397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.873772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.874651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.875165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.875437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.876171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.878442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.909109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 962/1938] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:47.910850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.922743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed4c3-3f26-481c-b699-dd12a85e51f2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:47.935452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:47.936085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:47.936439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:47.936624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:47.937591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:47.941006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:47.954417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a2f09dd1df44afb5f2d20486bf20cf', 'request_id': 'req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:47.963743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a2f09dd1df44afb5f2d20486bf20cf', 'request_id': 'req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:47.964980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:47.972626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 963/1939] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:47 2022] POST /identity/v3/users => generated 300 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:47.972626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:47.976493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a2f09dd1df44afb5f2d20486bf20cf', 'request_id': 'req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:47.984565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422bea5-e7e7-4a2a-afdb-5a2941f766bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.000232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] The token cache is being invalidate because project f6a2f09dd1df44afb5f2d20486bf20cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:48.000700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.001708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.004449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 977/1940] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:47 2022] DELETE /identity/v3/projects/f6a2f09dd1df44afb5f2d20486bf20cf => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:48.004779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.007032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.097092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.098513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.099037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.099149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.100096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.102313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3168e7-e5c7-4bd3-8e2b-1b9bb0865f4d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.111848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0e49d-01f3-459c-b6bd-f9ba29c28d72 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.116176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 964/1941] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:48.117392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.117746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.118131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.120375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 965/1942] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:14:48.121209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.121454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.122199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.122584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.123036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.123429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.126671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.127334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.138977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 978/1943] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:48.139981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.141710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.149958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8216-1c42-4dbe-88be-286c5a50138b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.154733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.155368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.155645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.155912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] PATH_INFO: `/v3/users/c2dcf70ad88d49d684ad08fabea29394` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.157031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2dcf70ad88d49d684ad08fabea29394)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.157523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc26e4-ba97-4274-b75a-8fe16f481f85 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.159203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.163176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.163777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.164069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.164328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.165002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.166596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e579-f8ad-4007-8071-52363a891c58 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.176995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 979/1944] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:48.178262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.178549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.178825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.202234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:48.204260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.224798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2dcf70ad88d49d684ad08fabea29394'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.232812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2dcf70ad88d49d684ad08fabea29394'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.245136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2dcf70ad88d49d684ad08fabea29394'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.246061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2dcf70ad88d49d684ad08fabea29394'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.271067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2dcf70ad88d49d684ad08fabea29394'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:48.281181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 966/1945] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:48 2022] DELETE /identity/v3/users/c2dcf70ad88d49d684ad08fabea29394 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:48.291924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.294878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.323400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:48.383649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 980/1946] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:14:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:48.384226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.385711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.400561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4d33f-5234-4920-9aea-742b7bce47c4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.418819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.419553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.419666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4556bd-c7d2-4b04-9d2e-3b85dff65aaa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.419836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.420118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] PATH_INFO: `/v3/projects/4039fec7c50f4b7fbbfc56373f64d639/users/a2c02f5c433049e5a8a9c3a836647681/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.430846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.432098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.432098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.432305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] PATH_INFO: `/v3/projects/895c75e1d39b4064aff03c8a33e52159/users/29bf7c4a206e44e08d330e5d24b966b0/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.440018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4039fec7c50f4b7fbbfc56373f64d639, user_id=a2c02f5c433049e5a8a9c3a836647681, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.444441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.452056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895c75e1d39b4064aff03c8a33e52159, user_id=29bf7c4a206e44e08d330e5d24b966b0, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.454868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa11ac-e477-46b5-b9af-5563025aa620 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.465244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 981/1947] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:48 2022] PUT /identity/v3/projects/4039fec7c50f4b7fbbfc56373f64d639/users/a2c02f5c433049e5a8a9c3a836647681/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:48.465244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.465244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.478213 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 967/1948] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:48 2022] PUT /identity/v3/projects/895c75e1d39b4064aff03c8a33e52159/users/29bf7c4a206e44e08d330e5d24b966b0/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:48.479474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.479831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.480172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.520928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] MFA Rules not processed for user `a2c02f5c433049e5a8a9c3a836647681`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:48.532875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.533079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:48.564872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c505-eb19-40d1-b0fb-9ad83e318014 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.611952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 968/1949] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:48.612638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.657580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.658220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.658496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.658762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.667954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.669562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.681572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 982/1950] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:48.682429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.695984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3f1f-b5ae-454e-a0f8-b38812252e5e service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.696631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f49580-466d-4577-8716-b244b7cdc3cc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.701073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.702042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.702532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.702997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.703122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.703948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.704316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.704595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.705183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.705846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.707183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.708811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.733161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 969/1951] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:48.737549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.755825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 983/1952] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:48.764713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.765216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c88a3-6848-4e3a-8fa5-69d3915e54fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.861302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:48.907366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab25ce-d918-4340-a339-10f981636067 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:48.912578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.913244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.913529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.913830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.914481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.917056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.920194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:48.921094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:48.921599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:48.922291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] PATH_INFO: `/v3/projects/895c75e1d39b4064aff03c8a33e52159/users/29bf7c4a206e44e08d330e5d24b966b0/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:48.924715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=895c75e1d39b4064aff03c8a33e52159, user_id=29bf7c4a206e44e08d330e5d24b966b0, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:48.928635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:48.948064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 970/1953] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:48 2022] PUT /identity/v3/projects/895c75e1d39b4064aff03c8a33e52159/users/29bf7c4a206e44e08d330e5d24b966b0/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:48.949083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.954842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 984/1954] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:48 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:48.955751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:48.965840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:48.981649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea764e-c65a-465b-a68f-edf24102a8bf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.044166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:49.057657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcffff5-e424-40da-9c8f-0abf7463b4d4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.065332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.066074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.066426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.067555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] PATH_INFO: `/v3/projects/6f43069e37d446b18fa2ff56370b0868` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.068828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f43069e37d446b18fa2ff56370b0868)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.071818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.094006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.095785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.096317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.096650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.097500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.100108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348e567-11cf-4b7b-bd59-43148c4ab70c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.116102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 985/1955] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:49.120503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884e11-2d8f-478d-bc36-956dcdc02381 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.120834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884e11-2d8f-478d-bc36-956dcdc02381 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.121229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71884e11-2d8f-478d-bc36-956dcdc02381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.161642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-71884e11-2d8f-478d-bc36-956dcdc02381 None None] MFA Rules not processed for user `29bf7c4a206e44e08d330e5d24b966b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:49.162920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-71884e11-2d8f-478d-bc36-956dcdc02381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.191257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f43069e37d446b18fa2ff56370b0868', 'request_id': 'req-391346c7-873f-4656-9ab4-39afe8a85bee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:49.206322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f43069e37d446b18fa2ff56370b0868', 'request_id': 'req-391346c7-873f-4656-9ab4-39afe8a85bee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:49.215015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f43069e37d446b18fa2ff56370b0868', 'request_id': 'req-391346c7-873f-4656-9ab4-39afe8a85bee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:49.245910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] The token cache is being invalidate because project 6f43069e37d446b18fa2ff56370b0868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:49.246210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:49.248654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:49.253037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 971/1956] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:48 2022] DELETE /identity/v3/projects/6f43069e37d446b18fa2ff56370b0868 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:49.255517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.257739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.322460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 986/1957] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:49.324133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.324832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.325364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.353933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:49.365907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:49.371803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.373250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391346c7-873f-4656-9ab4-39afe8a85bee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.381932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.383648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.383956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.384278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] PATH_INFO: `/v3/projects/180ff5b20fa1404c9c264042b1e4f451/users/423dc9b0930d46c29b3e12e26c7dad0f/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.400002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 987/1958] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:49.408389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.410056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.417157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180ff5b20fa1404c9c264042b1e4f451, user_id=423dc9b0930d46c29b3e12e26c7dad0f, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.417157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508db19-17c5-422c-a711-0774ae3c6984 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.442872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 972/1959] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:49 2022] PUT /identity/v3/projects/180ff5b20fa1404c9c264042b1e4f451/users/423dc9b0930d46c29b3e12e26c7dad0f/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:49.444094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.444384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.444657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.491672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:49.492909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.520986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a42be-0f02-4dfa-97c9-4286c46855db None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.565278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 973/1960] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:49.566178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.580598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4e19-915c-4be1-bcaa-b0c4dca72ffe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.596190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.598688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.599044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.599368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.600153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.602008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.615717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 974/1961] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:49.616586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.628896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1da6-7f9b-47ae-ba2b-606ed8576fc7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.634831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.636758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.637634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.637764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.638087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.638376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.638653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.639085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.639887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:49.640362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.642203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.653682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:49.655745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 975/1962] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:49.656753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.669155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f9af-f76b-495a-9cb9-2b0aeaa4c2a4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.674412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.675231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.675519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.675822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] PATH_INFO: `/v3/projects/180ff5b20fa1404c9c264042b1e4f451/users/423dc9b0930d46c29b3e12e26c7dad0f/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.677904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=180ff5b20fa1404c9c264042b1e4f451, user_id=423dc9b0930d46c29b3e12e26c7dad0f, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.680746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.693007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 976/1963] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:49 2022] PUT /identity/v3/projects/180ff5b20fa1404c9c264042b1e4f451/users/423dc9b0930d46c29b3e12e26c7dad0f/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:49.694282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.713859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.715336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23501f73-0b02-49ee-a4f0-47533e643cc7 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.726181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 988/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:49.727433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.727725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.727990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.768448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a326a-b877-43ab-933d-e90b43ada2a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.780120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.780765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.781044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.781732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.781889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] MFA Rules not processed for user `423dc9b0930d46c29b3e12e26c7dad0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:49.782977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.783133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.786682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.817158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 977/1965] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:49.821135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.822970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:49.864852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:49.918789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 989/1966] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:49.919661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:49.960176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf8c62-199e-4eda-bdf8-2788264302cf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.961950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ab143-09f2-40d9-9875-589b95c8fa8a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:49.969738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.970405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.970688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.970966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] PATH_INFO: `/v3/users/b18d8d413b15494ab4369490bf0ff9ad` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.972153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:49.972833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:49.973109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:49.973513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:49.974035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.975991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:49.979450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18d8d413b15494ab4369490bf0ff9ad)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:49.981464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.021026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 990/1967] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:49 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:50.021026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.022803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.070142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18d8d413b15494ab4369490bf0ff9ad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.077565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18d8d413b15494ab4369490bf0ff9ad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.086660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18d8d413b15494ab4369490bf0ff9ad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.087692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18d8d413b15494ab4369490bf0ff9ad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.101156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68dd81-4f22-48a6-8913-28fd10ece116 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.102423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18d8d413b15494ab4369490bf0ff9ad'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.116291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 978/1968] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:49 2022] DELETE /identity/v3/users/b18d8d413b15494ab4369490bf0ff9ad => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:50.117278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.117880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.118601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.118702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.118876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.119152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.120945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.157289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:50.186503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868b49d-c2bc-4931-8015-bf52d1bc557a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.191124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.192267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.192546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.192814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.193931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.194183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.195795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18a95f-8523-41ed-991b-924c2c33f951 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.196124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.209174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 979/1969] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:50.209354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 991/1970] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:50.209461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.209538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.209884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.227153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.240868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308d36e-a0e1-42a9-83da-d07565929bfc service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.243357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:50.244576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.255446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.255713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 980/1971] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:50.256868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.258258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.258776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.261942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.263642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.272403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.274360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.297474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 992/1972] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:50.300306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.314064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1b8d4-c8dd-49de-a584-0480846f3505 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.341547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.342240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.342525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.342803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] PATH_INFO: `/v3/projects/d49c075e1b8040568bcfb1ef1fccacb6/users/0a5a694e6e4040bc87d48ffc26256f42/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.354045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:50.363006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49c075e1b8040568bcfb1ef1fccacb6, user_id=0a5a694e6e4040bc87d48ffc26256f42, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.366597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.394044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 993/1973] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:50 2022] PUT /identity/v3/projects/d49c075e1b8040568bcfb1ef1fccacb6/users/0a5a694e6e4040bc87d48ffc26256f42/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:50.394044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.403159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.408826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299352b-5405-45e9-93cc-fccd76d05d57 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.477516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.478853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.479159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.479647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.491636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.491820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58374d-074e-4863-a541-4b3a7e6bec51 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.493401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1e345-6988-4ecd-a9b7-9f168e64e6b1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.504685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.505527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.505813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.506413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] PATH_INFO: `/v3/users/a2c02f5c433049e5a8a9c3a836647681` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.507737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c02f5c433049e5a8a9c3a836647681)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.510406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 981/1974] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:50.511586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.511875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.512149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.518149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.560147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] MFA Rules not processed for user `0a5a694e6e4040bc87d48ffc26256f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:50.562478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.609585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c02f5c433049e5a8a9c3a836647681'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.611331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c02f5c433049e5a8a9c3a836647681'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.626503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c02f5c433049e5a8a9c3a836647681'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.627551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c02f5c433049e5a8a9c3a836647681'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.644424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d85f46d5-5cdd-4f31-9de2-c9aa102ec151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c02f5c433049e5a8a9c3a836647681'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:50.678083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 994/1975] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:50 2022] DELETE /identity/v3/users/a2c02f5c433049e5a8a9c3a836647681 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:50.683200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.684096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.684490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.699281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 982/1976] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:50.703891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.708298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.724681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:50.727262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:50.731778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747952f-34f5-4584-a162-ba9f3a8a19a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:50.739941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:50.956334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:50.957021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:50.957413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:50.957694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:50.958344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:50.960882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:50.964815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 995/1977] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:50.972396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:50.974550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.012300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 983/1978] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:50 2022] POST /identity/v3/projects => generated 365 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:51.016712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.026699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:51.064480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10671-831b-46f8-a7fb-8ab89fd36a51 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.095300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f5f-17a4-4d20-8fa2-7ae68f133d69 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.101820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.102954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.102954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.102954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.103606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.106514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.128632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.130041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.130330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.130608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.143803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.145333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.154502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 984/1979] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:51 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:51.155425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.157411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.159502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 996/1980] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:51.161583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.171875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd074041-a6d2-4ed4-96c6-ad1399674fa3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.178077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.178929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.179042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e51ef1-c92e-4f45-ab85-d08900a2cfd2 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.179555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.179905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] PATH_INFO: `/v3/projects/cc6ce0d5670640368fd4b1745f3c543c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.181251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6ce0d5670640368fd4b1745f3c543c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.183714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.192770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.193348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.193671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.193950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.194710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.196338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.218394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 997/1981] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:51.223391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.236883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c247283-c2e8-4ccd-b0b4-2e67af47ee82 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.247051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.247773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.248065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.248539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] PATH_INFO: `/v3/projects/b2a75ea664494d68a335a75ab5e27136/users/291f66f39d3f429091a48df5d85d87ce/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.263718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6ce0d5670640368fd4b1745f3c543c', 'request_id': 'req-38a1f265-f125-420f-a6fb-83d56eb89ace'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.276015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a75ea664494d68a335a75ab5e27136, user_id=291f66f39d3f429091a48df5d85d87ce, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.276707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6ce0d5670640368fd4b1745f3c543c', 'request_id': 'req-38a1f265-f125-420f-a6fb-83d56eb89ace'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.279016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.290442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6ce0d5670640368fd4b1745f3c543c', 'request_id': 'req-38a1f265-f125-420f-a6fb-83d56eb89ace'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.299405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 998/1982] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:51 2022] PUT /identity/v3/projects/b2a75ea664494d68a335a75ab5e27136/users/291f66f39d3f429091a48df5d85d87ce/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:51.304425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.328025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] The token cache is being invalidate because project cc6ce0d5670640368fd4b1745f3c543c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:51.328467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.329323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.331250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 985/1983] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:51 2022] DELETE /identity/v3/projects/cc6ce0d5670640368fd4b1745f3c543c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:51.334945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.336442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.366828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:51.420228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca7b3-212b-4455-9ca2-3540213b6ad4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.425581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.426422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.426768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.427108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.428191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.429941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.438929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1f265-f125-420f-a6fb-83d56eb89ace admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.445456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 999/1984] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:51.446567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.447838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.449280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.449796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.450086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.450366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] PATH_INFO: `/v3/users/29bf7c4a206e44e08d330e5d24b966b0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.451657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bf7c4a206e44e08d330e5d24b966b0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.453955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.465625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2acb06-0deb-4b5a-a503-66a76a077bfb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.473375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.474102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.474490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.474838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] PATH_INFO: `/v3/projects/b2a75ea664494d68a335a75ab5e27136/users/291f66f39d3f429091a48df5d85d87ce/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.477564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a75ea664494d68a335a75ab5e27136, user_id=291f66f39d3f429091a48df5d85d87ce, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.480408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ffe32f-3288-443c-aa22-f03dacd690dc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.480974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:51.505408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1000/1985] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:51 2022] PUT /identity/v3/projects/b2a75ea664494d68a335a75ab5e27136/users/291f66f39d3f429091a48df5d85d87ce/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:51.517607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.518061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.518513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.528885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bf7c4a206e44e08d330e5d24b966b0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.538751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bf7c4a206e44e08d330e5d24b966b0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.548131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bf7c4a206e44e08d330e5d24b966b0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.548858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bf7c4a206e44e08d330e5d24b966b0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.565763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bf7c4a206e44e08d330e5d24b966b0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.565994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] MFA Rules not processed for user `291f66f39d3f429091a48df5d85d87ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:51.567708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.575843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 986/1986] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:51 2022] DELETE /identity/v3/users/29bf7c4a206e44e08d330e5d24b966b0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:51.577538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.580720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.666228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1001/1987] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:51.676352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.678331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.745351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93ba1-568c-43b0-8e54-737a69a0f5c3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.762321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.763504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.763857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.764204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] PATH_INFO: `/v3/projects/4039fec7c50f4b7fbbfc56373f64d639` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.766261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4039fec7c50f4b7fbbfc56373f64d639)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.767896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.777178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8b15b-fa1a-44fe-b418-62feaf29d69b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.831727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4039fec7c50f4b7fbbfc56373f64d639', 'request_id': 'req-e89665a9-c74a-4074-9377-f200f35306e1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.840245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4039fec7c50f4b7fbbfc56373f64d639', 'request_id': 'req-e89665a9-c74a-4074-9377-f200f35306e1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.846215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4039fec7c50f4b7fbbfc56373f64d639', 'request_id': 'req-e89665a9-c74a-4074-9377-f200f35306e1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.849141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.849905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.850456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.850761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.861382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] The token cache is being invalidate because project 4039fec7c50f4b7fbbfc56373f64d639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:51.861680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.862478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:51.864950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 987/1988] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:51 2022] DELETE /identity/v3/projects/4039fec7c50f4b7fbbfc56373f64d639 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:51.867284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:51.868770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:51.870209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:51.911314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:51.945273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.946723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:51.959572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89665a9-c74a-4074-9377-f200f35306e1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:51.960963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1002/1989] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:14:51.965301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:51.966202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:51.966484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:51.966895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] PATH_INFO: `/v3/users/423dc9b0930d46c29b3e12e26c7dad0f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:51.968067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=423dc9b0930d46c29b3e12e26c7dad0f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:51.971120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.040351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423dc9b0930d46c29b3e12e26c7dad0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.078783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423dc9b0930d46c29b3e12e26c7dad0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.091944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.093742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.097755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423dc9b0930d46c29b3e12e26c7dad0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.098912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423dc9b0930d46c29b3e12e26c7dad0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.124643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dffc1-af5a-451c-b8aa-5a720da86da3 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:52.134521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423dc9b0930d46c29b3e12e26c7dad0f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.138417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:52.140037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.140431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.142938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] PATH_INFO: `/v3/projects/895c75e1d39b4064aff03c8a33e52159` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.148946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=895c75e1d39b4064aff03c8a33e52159)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:52.163309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.167814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 988/1990] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:51 2022] DELETE /identity/v3/users/423dc9b0930d46c29b3e12e26c7dad0f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:52.168862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.174241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.286713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895c75e1d39b4064aff03c8a33e52159', 'request_id': 'req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.305167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895c75e1d39b4064aff03c8a33e52159', 'request_id': 'req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.306083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7626e4-3802-48bf-abbf-fe5d66a92866 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:52.315545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895c75e1d39b4064aff03c8a33e52159', 'request_id': 'req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.344101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] The token cache is being invalidate because project 895c75e1d39b4064aff03c8a33e52159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:52.348384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.351203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d337b24-b6c7-4eb4-8890-41aa5fef5b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.359827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1003/1991] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:52 2022] DELETE /identity/v3/projects/895c75e1d39b4064aff03c8a33e52159 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:52.361763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1434-cdf2-4b8f-aef5-df0ab94071cb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.362032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1434-cdf2-4b8f-aef5-df0ab94071cb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.362314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1434-cdf2-4b8f-aef5-df0ab94071cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.391010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-79dd1434-cdf2-4b8f-aef5-df0ab94071cb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:52.392387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-79dd1434-cdf2-4b8f-aef5-df0ab94071cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.441958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:52.443024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.443531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.444900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.448911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.494330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:52.527283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1004/1992] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:52.537496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.544619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.547385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.598791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:52.600846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.612746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1005/1993] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:52.616954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.632173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:52.633944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.634974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b331-5287-46d2-bdc6-50272491f2bb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:52.657905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 989/1994] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:52.664183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:52.666267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.666773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.668099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.670233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:52.670793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:52.673918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.706680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1006/1995] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:52.711778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.735233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625005-f718-43b3-8eac-92f54e2e5098 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:52.748503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:52.757248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.757549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.757829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.758648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:52.760730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.820372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.822862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:52.852350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ca728-c382-4e37-aa29-be4d52da9fec admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:52.859813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:52.860577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:52.860931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:52.861320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] PATH_INFO: `/v3/projects/180ff5b20fa1404c9c264042b1e4f451` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:52.862529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=180ff5b20fa1404c9c264042b1e4f451)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:52.864750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:52.878419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:52.881664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 990/1996] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:52 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:52.886425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:52.969445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '180ff5b20fa1404c9c264042b1e4f451', 'request_id': 'req-37aa4713-b979-427a-9291-c12018a46f45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.983510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '180ff5b20fa1404c9c264042b1e4f451', 'request_id': 'req-37aa4713-b979-427a-9291-c12018a46f45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:52.994012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '180ff5b20fa1404c9c264042b1e4f451', 'request_id': 'req-37aa4713-b979-427a-9291-c12018a46f45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:53.014197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] The token cache is being invalidate because project 180ff5b20fa1404c9c264042b1e4f451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:53.014384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:53.015483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-37aa4713-b979-427a-9291-c12018a46f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:53.017892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1007/1997] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:52 2022] DELETE /identity/v3/projects/180ff5b20fa1404c9c264042b1e4f451 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:53.023024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43ebb-6a49-4046-a912-24ceee8ce313 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.023403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43ebb-6a49-4046-a912-24ceee8ce313 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.024196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43ebb-6a49-4046-a912-24ceee8ce313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.054394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-caa43ebb-6a49-4046-a912-24ceee8ce313 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:53.055682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-caa43ebb-6a49-4046-a912-24ceee8ce313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:53.081163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2713-e254-427c-a40e-5d799a301138 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.207797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.210991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.211482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.211943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.212819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.215825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.225434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1008/1998] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:53.234983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.235340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.235676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.255900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 991/1999] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:52 2022] POST /identity/v3/users => generated 291 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:53.258241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:53.259520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:53.264287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.266991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:53.272903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1009/2000] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:53.277199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.282724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdbeab-4d75-4e97-8f97-def9c389625c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.288934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.289600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.289882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.290453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.290809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.292615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.293227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a71668-bc37-4a7f-89ea-3752e4f85a87 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.299993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.301153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.301499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.301789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.302469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:53.302879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.305064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.317795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 992/2001] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:53.318843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.327388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1010/2002] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:53.328351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.338978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fd39d-338f-487e-b7e9-894cdbe9a9cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.345959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.346595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.347069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.348105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.348965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.350712 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.360019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17bfe39-503e-4aef-b03e-69dec193328c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.367676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.368550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.368837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.369134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.369783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.371693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.378074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 993/2003] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:53.382988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.400775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdf600-2a36-40c7-b7a8-736aaa047a72 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.407080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.408563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.408859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.409480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] PATH_INFO: `/v3/projects/8b08f687dd0543f0811426ba4e34332f/users/be4eb82e198d462ab6c1b306063e45d3/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.412269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1011/2004] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/projects => generated 361 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:53.417020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.420850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:53.430241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b08f687dd0543f0811426ba4e34332f, user_id=be4eb82e198d462ab6c1b306063e45d3, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.433908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb7f8-2057-410b-ad38-5c0cca60d26f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.456874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 994/2005] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:53 2022] PUT /identity/v3/projects/8b08f687dd0543f0811426ba4e34332f/users/be4eb82e198d462ab6c1b306063e45d3/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:53.467091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.467390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.467655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.485951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:53.492468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78a624-0b00-41ac-af81-a295e1d40272 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.501019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] MFA Rules not processed for user `be4eb82e198d462ab6c1b306063e45d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:53.502262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:53.559204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.560294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.560781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.561290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.562302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.565762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.607035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1012/2006] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/users => generated 281 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:53.611287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.625079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6486c71-d77e-470c-8a92-c2aa64421624 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.632480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.633411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.633856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.634208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.634963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.639926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.656017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 995/2007] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:53.656978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.663000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1013/2008] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:53.664128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.694147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193880a-d2a6-4a38-a9e9-8e49df764d15 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.709846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa8f86-619c-4c85-af33-5841749b67c2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.710244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.710883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.711166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.711436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.712227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.714317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.730591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.731213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.733864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.734436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.745253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.757088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:53.799050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 996/2009] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:53.800716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1014/2010] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:53.806845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.809403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:53.826830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce3417-479e-4e2d-97ff-e36321e1d46c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.960951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.962036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.962334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.963004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] PATH_INFO: `/v3/projects/91b64dc883d54b28b0d34d194ba5a076/users/eb14de5ec29e4a32a44731221d5455f1/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.963428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee352-9106-4c02-829d-58b39b3bacc1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:53.993530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:53.994751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:53.995190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:53.996137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:53.997421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:53.998961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b64dc883d54b28b0d34d194ba5a076, user_id=eb14de5ec29e4a32a44731221d5455f1, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.001799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594a88-00e3-42f3-a8b8-c3f1c5c0490b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.002852 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.028454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1015/2011] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:53 2022] PUT /identity/v3/projects/91b64dc883d54b28b0d34d194ba5a076/users/eb14de5ec29e4a32a44731221d5455f1/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:54.040713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.042183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.043019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.052096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 997/2012] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:53 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:54.053274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.065516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e93768-db23-424a-80f7-644d501b1947 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.126861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] MFA Rules not processed for user `eb14de5ec29e4a32a44731221d5455f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:54.128345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.174858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:54.218616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.221793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.222088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.224681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.225434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.227640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.239084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:54.268382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 998/2013] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:54.274353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.287586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac708e7-dce4-4197-9688-796e952f6d51 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.295349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.296430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.296667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.297029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] PATH_INFO: `/v3/projects/7f53b41288894fc18657572564a0a4db/users/efdecf017ca74bfc9bdb8abb949382af/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.329986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1016/2014] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:54.330709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f53b41288894fc18657572564a0a4db, user_id=efdecf017ca74bfc9bdb8abb949382af, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.334347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.359574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 999/2015] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:54 2022] PUT /identity/v3/projects/7f53b41288894fc18657572564a0a4db/users/efdecf017ca74bfc9bdb8abb949382af/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:54.366763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.367612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.368802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.379824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06bbcc-224f-48a7-8740-799dbda65bcc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.473637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e60ed-7ec8-4054-9412-71ffa24f311a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.479640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.480258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.480844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.481144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.488775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.490394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.490749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.491095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.492591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.493683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.509313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1017/2016] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:54.513234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.526700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d780c-98f6-432b-bcfc-0fbf480d3a41 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.533434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.534342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.534806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.535321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] PATH_INFO: `/v3/projects/7f53b41288894fc18657572564a0a4db/users/efdecf017ca74bfc9bdb8abb949382af/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.537859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f53b41288894fc18657572564a0a4db, user_id=efdecf017ca74bfc9bdb8abb949382af, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.542049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e260b-79ab-484e-97f1-eef6b633e359 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.563648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.564965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1018/2017] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:54 2022] PUT /identity/v3/projects/7f53b41288894fc18657572564a0a4db/users/efdecf017ca74bfc9bdb8abb949382af/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:54.565143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.576694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.576907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.577161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.577940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1000/2018] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:54.612427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.625882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] MFA Rules not processed for user `efdecf017ca74bfc9bdb8abb949382af`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:54.627264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.630244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1f4f4-b4ef-49b0-b7dc-2cb078bababb service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.705440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.706272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.706615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.706936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] PATH_INFO: `/v3/users/eb14de5ec29e4a32a44731221d5455f1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.708932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb14de5ec29e4a32a44731221d5455f1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:54.711231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:54.727753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:54.795203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb14de5ec29e4a32a44731221d5455f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:54.803040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb14de5ec29e4a32a44731221d5455f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:54.813257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb14de5ec29e4a32a44731221d5455f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:54.814211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb14de5ec29e4a32a44731221d5455f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:54.817100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1019/2019] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:54.829421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.830711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d7bd861-29ed-410d-b1cb-a46bb19c182c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb14de5ec29e4a32a44731221d5455f1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:54.830822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.837394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1001/2020] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:54 2022] DELETE /identity/v3/users/eb14de5ec29e4a32a44731221d5455f1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:54.842241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.842602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.842808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.873707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:54.875227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.896991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d640099-977c-46ab-baaf-aef3ba03b09b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:54.900535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:54.944346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1002/2021] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:54.948700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:54.949364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:54.949638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:54.949909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:54.951676 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:54.956815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:54.959092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.019854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e18151-9dfa-4cb3-ad49-7f38224c68da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.023641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.024000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.024287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.024564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.024836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.025486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.036901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.040748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1020/2022] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:55.040959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.050480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1003/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:55.138503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.139920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.150583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0292e-ed23-4827-88bb-356395a4152e service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.154539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.155200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.155516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.155797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.157725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.235320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.236973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd679017-1df9-4eb4-b337-5a2d68ab79d8 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.249871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1021/2024] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:55.372867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.374459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.409250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a83e4-0718-439e-80fc-28da7a6d0db5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.421846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.422531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.422863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.423140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] PATH_INFO: `/v3/projects/91b64dc883d54b28b0d34d194ba5a076` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.424314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b64dc883d54b28b0d34d194ba5a076)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.430203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.469836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:55.495695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b64dc883d54b28b0d34d194ba5a076', 'request_id': 'req-5229fe90-890c-498a-ac79-bb6c728bfbb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.503445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b64dc883d54b28b0d34d194ba5a076', 'request_id': 'req-5229fe90-890c-498a-ac79-bb6c728bfbb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.511169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b64dc883d54b28b0d34d194ba5a076', 'request_id': 'req-5229fe90-890c-498a-ac79-bb6c728bfbb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.526621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] The token cache is being invalidate because project 91b64dc883d54b28b0d34d194ba5a076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:55.526918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.527594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.529106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1004/2025] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:55 2022] DELETE /identity/v3/projects/91b64dc883d54b28b0d34d194ba5a076 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:55.534062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12b15-47c0-4b13-9865-c47ffb2b30cb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.535712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12b15-47c0-4b13-9865-c47ffb2b30cb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.536056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f12b15-47c0-4b13-9865-c47ffb2b30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.563099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c7f12b15-47c0-4b13-9865-c47ffb2b30cb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:55.564415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c7f12b15-47c0-4b13-9865-c47ffb2b30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.652010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:55.664261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1022/2026] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:55.669273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.670692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.671704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.671994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.672274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.685744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fe90-890c-498a-ac79-bb6c728bfbb4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.691088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.692314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.693162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.693162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] PATH_INFO: `/v3/users/be4eb82e198d462ab6c1b306063e45d3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.694071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4eb82e198d462ab6c1b306063e45d3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.696021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.698308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:55.699536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.707459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1023/2027] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:14:55.711877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.721462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3b287-a8f5-493d-9533-10dc5fff5343 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.726229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.726822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.727098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.727364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.727996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:55.728294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.729909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.742959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4eb82e198d462ab6c1b306063e45d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.744763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1024/2028] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:14:55.749999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.759016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4eb82e198d462ab6c1b306063e45d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.766905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd583f-cb34-4655-9290-f787cb159136 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.773068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.773692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.773974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.774247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.774883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.776904 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.777335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4eb82e198d462ab6c1b306063e45d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.777991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4eb82e198d462ab6c1b306063e45d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.794848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c6eef72-f46e-4bf8-994b-d17f80403796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4eb82e198d462ab6c1b306063e45d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:55.804003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1005/2029] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:55 2022] DELETE /identity/v3/users/be4eb82e198d462ab6c1b306063e45d3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:55.810790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.810790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.810790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.822287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1025/2030] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:55 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:55.826100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.828377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.831359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:55.833143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.922662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edee643-9e6b-4abf-a10a-604316316fb4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:55.930348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:55.931042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:55.931398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:55.931744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:55.932450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:55.943535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:55.953633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1006/2031] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:55.957811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:55.959708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:55.986421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1026/2032] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:55 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:55.993060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.004678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bcf3e-d8b9-4b72-9860-4d71c11babbc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.010758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.011511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.011874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.012006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:56.012839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.013603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.015727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.037865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1027/2033] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:56.042534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.067762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a679-ef67-4061-8398-670c8c3f5f91 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.077107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dd00e-b62b-4af8-8962-ed45631e963d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.083527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.084437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.084734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.085428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.085733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.086117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.086388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.086642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.086917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.087733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.100610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.102038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e264257-b5b7-4066-ba17-c1ec91067631 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.112947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1028/2034] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:56.112947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.124096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1007/2035] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:56.126368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33939ed-5d46-458f-b863-9a10367d61c3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.133476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.134721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.134881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.135271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] PATH_INFO: `/v3/projects/b35878884a7c4158943637adbdc232a5/users/66b3d82437f84321a4e0dec288d46828/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.168207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35878884a7c4158943637adbdc232a5, user_id=66b3d82437f84321a4e0dec288d46828, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.172978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.194607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1029/2036] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:56 2022] PUT /identity/v3/projects/b35878884a7c4158943637adbdc232a5/users/66b3d82437f84321a4e0dec288d46828/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:56.205567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.205856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.206132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.240116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] MFA Rules not processed for user `66b3d82437f84321a4e0dec288d46828`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:56.241434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:56.337769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1008/2037] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:56.346499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.348538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:56.425533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e08a0-bcef-492b-9957-f3a1623113cb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.429895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.430587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.430850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.431116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.440224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.442312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.448306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:56.461545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1030/2038] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:56.588634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.588634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:56.695481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9a781-3c0e-4503-b525-4d61b2e3c7d1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.702446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.703196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.704055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.704321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] PATH_INFO: `/v3/projects/8b08f687dd0543f0811426ba4e34332f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.705902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b08f687dd0543f0811426ba4e34332f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.708414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:56.792638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b08f687dd0543f0811426ba4e34332f', 'request_id': 'req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:56.813515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b08f687dd0543f0811426ba4e34332f', 'request_id': 'req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:56.829425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b08f687dd0543f0811426ba4e34332f', 'request_id': 'req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:56.829425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:56.850740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] The token cache is being invalidate because project 8b08f687dd0543f0811426ba4e34332f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:56.851036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:56.852075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:56.856588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1009/2039] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:56 2022] DELETE /identity/v3/projects/8b08f687dd0543f0811426ba4e34332f => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:56.860522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:56.883429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f700c-4bed-4c89-933b-4434a6645884 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:56.971757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:56.972009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:56.972378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:56.973752 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] PATH_INFO: `/v3/users/efdecf017ca74bfc9bdb8abb949382af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:56.975289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdecf017ca74bfc9bdb8abb949382af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:56.977665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:57.055948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdecf017ca74bfc9bdb8abb949382af'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.056836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdecf017ca74bfc9bdb8abb949382af'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.071175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdecf017ca74bfc9bdb8abb949382af'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.071927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdecf017ca74bfc9bdb8abb949382af'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.084806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cdfb1367-bb42-4d82-865e-f9919e4d92ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdecf017ca74bfc9bdb8abb949382af'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.140715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1031/2040] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:56 2022] DELETE /identity/v3/users/efdecf017ca74bfc9bdb8abb949382af => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:57.611202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:57.612698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:57.629949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f28e5-6bc6-4b71-80d0-8c4011d1c077 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:57.700365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:57.701257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:57.701555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:57.701898 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] PATH_INFO: `/v3/projects/7f53b41288894fc18657572564a0a4db` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:57.703249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f53b41288894fc18657572564a0a4db)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:57.705341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:57.800611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f53b41288894fc18657572564a0a4db', 'request_id': 'req-e05f9d91-c6d5-4114-a117-7094b5840033'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.811920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f53b41288894fc18657572564a0a4db', 'request_id': 'req-e05f9d91-c6d5-4114-a117-7094b5840033'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.826983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f53b41288894fc18657572564a0a4db', 'request_id': 'req-e05f9d91-c6d5-4114-a117-7094b5840033'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.844464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] The token cache is being invalidate because project 7f53b41288894fc18657572564a0a4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:14:57.844626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.845320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e05f9d91-c6d5-4114-a117-7094b5840033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:57.847453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1010/2041] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:57 2022] DELETE /identity/v3/projects/7f53b41288894fc18657572564a0a4db => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:57.854217 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:57.854669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:57.854990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:57.880035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:57.881656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:57.954921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1032/2042] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:57.962917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:57.963188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:57.963467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:57.984455 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:57.985694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:57.994705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1011/2043] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:57.998877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.010521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc742d1-ccc0-4041-8597-827e61b7c35f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.019175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.020216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.020705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.021166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.022604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:14:58.023088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.028454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.041159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1033/2044] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:14:58.046287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.057517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06161016-1afb-4bba-9e6c-02d6a77c9ea9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.062856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.063639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.064169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.064558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.065554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.068293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.099500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1012/2045] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:58 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:58.104398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.116907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ad17a-33ae-4210-913e-44a2d3001677 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.121451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:58.199881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.201894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.201894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.201894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.202398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.204508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.232323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1034/2046] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:58 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:58.236388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.250349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c23e-1492-49e4-9e40-18d41fea4af5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.258388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.259495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.259850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.260411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.261543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.265494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.283388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1013/2047] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:58.288677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.303902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d06da-1f1c-4d90-8ae2-af926c82f9a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.310306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.311026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.311305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.311625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.312351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.314277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b181e-d633-4f8b-a350-7294f0a6aeef admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.336431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1035/2048] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:58.337919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.348787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cec86d-13c9-4b12-8f73-9d0549ff0f7d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.354873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.355609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.355883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.356188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] PATH_INFO: `/v3/projects/cb3e952337bc40ce8f1013dede7a9e6a/users/bca484baf3d047e2b0b8681edf089793/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.398501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3e952337bc40ce8f1013dede7a9e6a, user_id=bca484baf3d047e2b0b8681edf089793, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.401669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.421426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1014/2049] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:58 2022] PUT /identity/v3/projects/cb3e952337bc40ce8f1013dede7a9e6a/users/bca484baf3d047e2b0b8681edf089793/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:58.436035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.437058 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.438280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.478544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] MFA Rules not processed for user `bca484baf3d047e2b0b8681edf089793`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:58.479702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:58.576657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1036/2050] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:58.583730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.607695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:58.738943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d29fd-b824-4b27-856e-1f29825a5abd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.757376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.758263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.758596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.759094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.759866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.764118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.826113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.833277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:58.856082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1015/2051] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:58.860803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:58.862441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb288-6a73-439d-879c-b6dc33464466 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:58.971460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:58.972321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:58.973486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:58.973934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] PATH_INFO: `/v3/users/0a5a694e6e4040bc87d48ffc26256f42` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:58.975258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5a694e6e4040bc87d48ffc26256f42)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:58.977488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:58.998769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403996a-35c9-4193-80bb-9150a37c3445 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.002257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:59.011584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.012717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.013021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.013562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.014448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.018726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.069521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1016/2052] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:58 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:59.074616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.075696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5a694e6e4040bc87d48ffc26256f42'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:59.078009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5a694e6e4040bc87d48ffc26256f42'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:59.089602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4a52-088d-4c0f-a2d3-10b7510ef806 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.095131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5a694e6e4040bc87d48ffc26256f42'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:59.096293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5a694e6e4040bc87d48ffc26256f42'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:59.097499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.098774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.100117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.100461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.106457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.108990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.116685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5221fcb7-1a01-4492-8a8f-b7583e653a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5a694e6e4040bc87d48ffc26256f42'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:14:59.128105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1017/2053] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:59.141571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.143668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:59.165706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bb2d-6a30-418a-81c4-b9fb2f0791ed admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.171140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.171875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.172221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.172558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.173301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.174946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.179372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1037/2054] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:14:58 2022] DELETE /identity/v3/users/0a5a694e6e4040bc87d48ffc26256f42 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:14:59.185502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.185801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.186079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.193585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1018/2055] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:14:59.197739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.210020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:59.211398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:59.211668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0a4ff-76ef-4e9f-ae28-fa60d922d493 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.304429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.305613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.306183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.306780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] PATH_INFO: `/v3/projects/1ac3e444ba43484eb161c1d8fcf0e8c6/users/d2ebd8d193124f14b9ad53bf6b7aeed3/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.309446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1038/2056] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:59.321274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.323451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:59.339198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac3e444ba43484eb161c1d8fcf0e8c6, user_id=d2ebd8d193124f14b9ad53bf6b7aeed3, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.345485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45d6eb-4712-40b4-9323-c8dce8d892d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.350000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:59.390643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1019/2057] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:59 2022] PUT /identity/v3/projects/1ac3e444ba43484eb161c1d8fcf0e8c6/users/d2ebd8d193124f14b9ad53bf6b7aeed3/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:59.402088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.402510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.402884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.415931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fddf2-3835-4e6b-b806-a5109c6e63af None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.475204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] MFA Rules not processed for user `d2ebd8d193124f14b9ad53bf6b7aeed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:14:59.476419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:14:59.496481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.497351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.497773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.498188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.503007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:14:59.557921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1020/2058] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:14:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:14:59.562337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.582701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.585841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.599498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1039/2059] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:14:59.643961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369eb3b-d5f4-45dc-ab74-ead9c9b92cd2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.653366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.654008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.654288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.654564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.655205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.657111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.701626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1021/2060] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:14:59 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:14:59.705382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.715172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd346947-ec0c-4e53-8391-1800fe690760 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.795991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.796603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.796877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.797164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.797796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.799806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.832636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1040/2061] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:14:59 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:14:59.836608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.848353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a03a35-3e14-474b-a11c-291918ff55ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.853071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.853846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.854123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.854395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.855027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.856654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.869211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1022/2062] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:14:59.872149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.883101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86aace8-f657-4c55-a0b8-cbb82abda6e5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.890963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:14:59.891710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:14:59.892056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:14:59.892400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] PATH_INFO: `/v3/projects/19faa3a42ef8456f869de1ebcf625944/users/3157b4ceb3724e8e90c3f88ea66fee9a/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:14:59.907977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19faa3a42ef8456f869de1ebcf625944, user_id=3157b4ceb3724e8e90c3f88ea66fee9a, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:14:59.911321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:14:59.928606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1041/2063] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:14:59 2022] PUT /identity/v3/projects/19faa3a42ef8456f869de1ebcf625944/users/3157b4ceb3724e8e90c3f88ea66fee9a/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:14:59.932392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.945264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee489c-3fd9-4844-b6cc-cd377efb06dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:14:59.971788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:14:59.974601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.034751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:00.043788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.044934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.045385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.045848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.046959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.048991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.060770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1023/2064] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:00.064911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:00.072786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36954-a456-40b4-a152-0a49267aa621 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:00.075323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1a5a-f12b-4ac8-a01b-ca1db8eb05b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:00.078204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.078932 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.079289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.079738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] PATH_INFO: `/v3/projects/d49c075e1b8040568bcfb1ef1fccacb6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.080981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49c075e1b8040568bcfb1ef1fccacb6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.081575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.083104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.083443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.083732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.083844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] PATH_INFO: `/v3/projects/19faa3a42ef8456f869de1ebcf625944/users/3157b4ceb3724e8e90c3f88ea66fee9a/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.085878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19faa3a42ef8456f869de1ebcf625944, user_id=3157b4ceb3724e8e90c3f88ea66fee9a, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.088747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec9866-98fe-4f13-9b94-a6642977f6b4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.093772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:00.101902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1024/2065] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:00 2022] PUT /identity/v3/projects/19faa3a42ef8456f869de1ebcf625944/users/3157b4ceb3724e8e90c3f88ea66fee9a/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:00.112865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.113463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.113798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.146608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49c075e1b8040568bcfb1ef1fccacb6', 'request_id': 'req-ed48d226-d83e-4a86-aee6-d96206ac9c66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.147682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] MFA Rules not processed for user `3157b4ceb3724e8e90c3f88ea66fee9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:00.149188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.155599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49c075e1b8040568bcfb1ef1fccacb6', 'request_id': 'req-ed48d226-d83e-4a86-aee6-d96206ac9c66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.163063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49c075e1b8040568bcfb1ef1fccacb6', 'request_id': 'req-ed48d226-d83e-4a86-aee6-d96206ac9c66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.176081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] The token cache is being invalidate because project d49c075e1b8040568bcfb1ef1fccacb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:00.176504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.177433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.179222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1042/2066] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:14:59 2022] DELETE /identity/v3/projects/d49c075e1b8040568bcfb1ef1fccacb6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:00.182712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:00.184276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.225967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1025/2067] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:00.240435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:00.241849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.268224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48d226-d83e-4a86-aee6-d96206ac9c66 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:00.274105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.274746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.275142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.275412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] PATH_INFO: `/v3/users/291f66f39d3f429091a48df5d85d87ce` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.276637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291f66f39d3f429091a48df5d85d87ce)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.278736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.310946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036ab8b-f456-4645-8efa-ba4f3a35cdc7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:00.331285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.332721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.333137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.333541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.338517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.339839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291f66f39d3f429091a48df5d85d87ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.340636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291f66f39d3f429091a48df5d85d87ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.351847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291f66f39d3f429091a48df5d85d87ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.352742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291f66f39d3f429091a48df5d85d87ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.364658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291f66f39d3f429091a48df5d85d87ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.392510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1043/2068] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:00 2022] DELETE /identity/v3/users/291f66f39d3f429091a48df5d85d87ce => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:00.419729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.421217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb53be-4669-41f7-9342-e19716c14123 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.432274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1026/2069] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:00.771166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:00.772952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:00.796696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:00.891606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47646ddb-c734-4abd-99eb-08bdafd43e30 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:00.897138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:00.898332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:00.899427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:00.900019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] PATH_INFO: `/v3/projects/b2a75ea664494d68a335a75ab5e27136` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:00.901833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a75ea664494d68a335a75ab5e27136)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:00.904304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:00.966830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a75ea664494d68a335a75ab5e27136', 'request_id': 'req-41cec73c-967d-44b6-84f9-6d503f46a2da'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.977539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a75ea664494d68a335a75ab5e27136', 'request_id': 'req-41cec73c-967d-44b6-84f9-6d503f46a2da'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.983917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a75ea664494d68a335a75ab5e27136', 'request_id': 'req-41cec73c-967d-44b6-84f9-6d503f46a2da'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.996486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] The token cache is being invalidate because project b2a75ea664494d68a335a75ab5e27136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:00.996780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.997440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41cec73c-967d-44b6-84f9-6d503f46a2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:00.999187 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1044/2070] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:00 2022] DELETE /identity/v3/projects/b2a75ea664494d68a335a75ab5e27136 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:01.005355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.005843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.005936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.026508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:01.027730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:01.041326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:01.090924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1027/2071] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:01.098919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.099202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.099476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.120066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:01.121310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:01.129803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1045/2072] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:01.134383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.143922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1ad9a-17cd-4295-a84a-ac8ea16a4d00 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.148496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.149193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.149751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.150027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.150663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:01.150960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.152582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.164569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1028/2073] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:01.168959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.179085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886beed6-a138-4dac-af45-eb95081cf963 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.183707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.184302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.184580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.184881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.185550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.187435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.215444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1046/2074] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:01 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:01.219326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.230516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16db55d-37c3-43cd-bf41-c7a114563eb2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.290217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.290557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.290876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.291151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.291902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.294469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.374433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1029/2075] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:01 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:01.375703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.388253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437da56d-5d2e-4bd4-8a6e-87869c2e494c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.393214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.393852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.394132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.394408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.395070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.396708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.410836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1047/2076] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:01.414958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.425452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f2aa-dc6b-47a8-b6dc-0f94b165fc29 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.430807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.431393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.431675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.431950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.432593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.434262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156a0993-bc58-4298-8e47-4d7c9e3c504b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.450731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1030/2077] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:01.452964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.469418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c67e0c-3309-4742-b147-5e816c8ae89f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.474959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.475697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.475980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.476261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] PATH_INFO: `/v3/projects/8e9f3629cb0e4cf393ddf8942c5a6946/users/83123e2a0aaa4f17b2f3c7dd70572658/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.491903 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9f3629cb0e4cf393ddf8942c5a6946, user_id=83123e2a0aaa4f17b2f3c7dd70572658, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.494609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.507813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1048/2078] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:01 2022] PUT /identity/v3/projects/8e9f3629cb0e4cf393ddf8942c5a6946/users/83123e2a0aaa4f17b2f3c7dd70572658/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:01.518964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.519248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.519518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.548265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] MFA Rules not processed for user `83123e2a0aaa4f17b2f3c7dd70572658`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:01.549532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:01.622038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1031/2079] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:01.633563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:01.635289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:01.682021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:01.695176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc227-d8c5-4c07-a901-55fb4834dd39 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:01.698883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:01.699482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:01.699762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:01.700035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:01.708943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:01.710400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:01.719068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1049/2080] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:03.164334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:03.173707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd08f-d294-4b66-a9b7-90efe7b20dfc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:03.177867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:03.178463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:03.178737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:03.179027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:03.180720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:03.194305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:03.239498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:03.240916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a070f59-1677-49d4-8135-8b71cc219119 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:03.250111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1032/2081] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:05.570121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:05.578594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4b12-8e9a-46bd-a149-062d6d1e8581 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:05.582177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:05.582758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:05.583034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:05.583304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:05.585020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:05.653516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:05.655141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:05.665860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1050/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:06.247477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610065-edf9-4277-b275-bc03c2c8f19f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:06.248030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610065-edf9-4277-b275-bc03c2c8f19f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:06.248030 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b610065-edf9-4277-b275-bc03c2c8f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:06.267016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7b610065-edf9-4277-b275-bc03c2c8f19f None None] MFA Rules not processed for user `83123e2a0aaa4f17b2f3c7dd70572658`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:06.268201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7b610065-edf9-4277-b275-bc03c2c8f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:06.275959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1033/2083] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:06.856627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:06.859125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:06.969657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aec3d-a337-4c37-8aef-cc8141d08742 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:06.974683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:06.975528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:06.976158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:06.976670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] PATH_INFO: `/v3/users/66b3d82437f84321a4e0dec288d46828` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:06.978442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b3d82437f84321a4e0dec288d46828)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:06.981219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:07.054055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b3d82437f84321a4e0dec288d46828'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.055018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b3d82437f84321a4e0dec288d46828'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.082116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b3d82437f84321a4e0dec288d46828'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.083477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b3d82437f84321a4e0dec288d46828'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.106627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b3d82437f84321a4e0dec288d46828'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.123885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:07.132131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1051/2084] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:06 2022] DELETE /identity/v3/users/66b3d82437f84321a4e0dec288d46828 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:07.135438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.135745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.136074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.160501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:07.162632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:07.236692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1034/2085] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:07.248400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:07.250564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:07.323959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a82c43-3eec-42e8-a3f8-f7b2d1234b2c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:07.327913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:07.328541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.328821 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.329236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.339672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:07.341103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99496251-415e-4b94-bd7f-a5c8c82334eb service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:07.351268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1052/2086] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:07.616288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:07.618296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:07.632989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b441b-c8f4-4b3c-b2f5-466ee1dd787c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:07.637961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:07.638597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.638873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.639149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] PATH_INFO: `/v3/projects/b35878884a7c4158943637adbdc232a5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.640263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35878884a7c4158943637adbdc232a5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:07.642221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:07.689938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35878884a7c4158943637adbdc232a5', 'request_id': 'req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.696037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35878884a7c4158943637adbdc232a5', 'request_id': 'req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.702197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35878884a7c4158943637adbdc232a5', 'request_id': 'req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.716159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] The token cache is being invalidate because project b35878884a7c4158943637adbdc232a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:07.716450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.717075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba1d4dd6-7768-47ae-bd66-03d49eeff2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:07.718578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1035/2087] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:07 2022] DELETE /identity/v3/projects/b35878884a7c4158943637adbdc232a5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:07.723419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.723691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.724645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.743709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:07.744836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:07.806503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1053/2088] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:07.814407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.814694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.814973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.836577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:07.837872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:07.846609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1036/2089] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:07.851869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:07.861744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a48fc-627a-4ff1-82f4-f2d31f0d7057 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:07.866291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:07.866879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.867160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.867433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.868074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:07.868376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:07.870001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:07.881952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1054/2090] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:07.886465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:07.896253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27291cf-0184-4fa2-b8b3-15c7c88c87ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:07.901006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:07.901636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:07.901914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:07.902191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:07.902817 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:07.904697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:07.935746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1037/2091] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:07 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:07.939696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:07.951300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d8e09-5b85-421f-b5e3-926f97cd2baf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:08.019421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:08.020135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.020476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.021452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.022221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:08.024239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:08.048240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1055/2092] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:07 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:08.052412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:08.057881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:08.065503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5995cc-12a8-43fe-95fa-3d8780415da7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:08.072739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:08.073486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.073833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.074175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.074902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:08.076624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:08.088976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1038/2093] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:08.092851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:08.102991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cde969-4240-470b-91a2-2538624d6cb1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:08.108012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:08.117785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.118470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.118757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.119543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:08.126119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7dac1-2c2e-4e0b-8511-c11f74d6dd20 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:08.145083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1056/2094] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:08.146864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:08.168581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f100a-9f10-468e-a209-264511412b5a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:08.173579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:08.174236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.174521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.174799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] PATH_INFO: `/v3/projects/e75c8121aefb4a0081cba2e575b8617b/users/22b9e583ba7d4e1ca531ec12f5c7c199/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.197664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75c8121aefb4a0081cba2e575b8617b, user_id=22b9e583ba7d4e1ca531ec12f5c7c199, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:08.200360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:08.221525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1039/2095] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:08 2022] PUT /identity/v3/projects/e75c8121aefb4a0081cba2e575b8617b/users/22b9e583ba7d4e1ca531ec12f5c7c199/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:08.232591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.232879 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.233191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.259435 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:08.276533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] MFA Rules not processed for user `22b9e583ba7d4e1ca531ec12f5c7c199`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:08.278087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:08.373047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1057/2096] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:08.381773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:08.383930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:08.466631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11c32-c942-4d10-8e0b-6baa0aa118b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:08.470281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:08.470890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:08.471169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:08.471434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:08.480713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:08.482228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c86dd-13fb-462f-9b99-56f12a9f0b53 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:08.492138 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1040/2097] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:10.881021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:10.883292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:10.958205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25588c2-8891-4b40-95e8-423a73578f40 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:10.963287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:10.963915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:10.964194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:10.964470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] PATH_INFO: `/v3/users/bca484baf3d047e2b0b8681edf089793` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:10.965626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bca484baf3d047e2b0b8681edf089793)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:10.967543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:11.019355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bca484baf3d047e2b0b8681edf089793'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.020022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bca484baf3d047e2b0b8681edf089793'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.028033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bca484baf3d047e2b0b8681edf089793'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.028812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bca484baf3d047e2b0b8681edf089793'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.036618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bca484baf3d047e2b0b8681edf089793'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.061771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1058/2098] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:10 2022] DELETE /identity/v3/users/bca484baf3d047e2b0b8681edf089793 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:11.066495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:11.067185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:11.067474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:11.088231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:11.089536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:11.147789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1041/2099] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:11.157868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:11.159437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:11.185263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:11.225760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e412990-99f7-4978-89d4-fe1154ccce5e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:11.229538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:11.230132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:11.230406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:11.230674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:11.239535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:11.240949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:11.250509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1059/2100] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:11.515994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:11.517551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:11.532400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d9a53-390a-49e2-8d3e-4a041cdcf620 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:11.537346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:11.538917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:11.539194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:11.539528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] PATH_INFO: `/v3/projects/cb3e952337bc40ce8f1013dede7a9e6a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:11.540668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3e952337bc40ce8f1013dede7a9e6a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:11.542702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:11.590132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3e952337bc40ce8f1013dede7a9e6a', 'request_id': 'req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.596206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3e952337bc40ce8f1013dede7a9e6a', 'request_id': 'req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.602197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3e952337bc40ce8f1013dede7a9e6a', 'request_id': 'req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.604615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:11.615911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] The token cache is being invalidate because project cb3e952337bc40ce8f1013dede7a9e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:11.616228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.619725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.621404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1042/2101] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:11 2022] DELETE /identity/v3/projects/cb3e952337bc40ce8f1013dede7a9e6a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:11.625519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:11.637124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db3057-5823-4622-b787-b01ade0609de service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:11.698119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:11.698816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:11.699096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:11.699386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] PATH_INFO: `/v3/users/d2ebd8d193124f14b9ad53bf6b7aeed3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:11.700581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ebd8d193124f14b9ad53bf6b7aeed3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:11.702540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:11.762090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ebd8d193124f14b9ad53bf6b7aeed3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.762761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ebd8d193124f14b9ad53bf6b7aeed3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.770938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ebd8d193124f14b9ad53bf6b7aeed3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.771646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ebd8d193124f14b9ad53bf6b7aeed3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.778991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ebd8d193124f14b9ad53bf6b7aeed3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:11.799207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1060/2102] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:11 2022] DELETE /identity/v3/users/d2ebd8d193124f14b9ad53bf6b7aeed3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:12.207004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:12.208825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:12.267297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc4e69-ca3d-43e2-947d-9205ad95d32e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:12.357010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:12.357736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:12.358018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:12.358290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] PATH_INFO: `/v3/projects/1ac3e444ba43484eb161c1d8fcf0e8c6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:12.361933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac3e444ba43484eb161c1d8fcf0e8c6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:12.364118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:12.416498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac3e444ba43484eb161c1d8fcf0e8c6', 'request_id': 'req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.423933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac3e444ba43484eb161c1d8fcf0e8c6', 'request_id': 'req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.430635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac3e444ba43484eb161c1d8fcf0e8c6', 'request_id': 'req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.446473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] The token cache is being invalidate because project 1ac3e444ba43484eb161c1d8fcf0e8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:12.446764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.447448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.448953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1043/2103] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:12 2022] DELETE /identity/v3/projects/1ac3e444ba43484eb161c1d8fcf0e8c6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:12.454085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:12.455575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:12.471194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cd0b4-9d40-4ac3-a33a-1c210e36641c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:12.510009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:12.536379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:12.537007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:12.537314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:12.537624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] PATH_INFO: `/v3/users/3157b4ceb3724e8e90c3f88ea66fee9a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:12.538749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3157b4ceb3724e8e90c3f88ea66fee9a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:12.540698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:12.589898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3157b4ceb3724e8e90c3f88ea66fee9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.590560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3157b4ceb3724e8e90c3f88ea66fee9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.597974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3157b4ceb3724e8e90c3f88ea66fee9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.598661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3157b4ceb3724e8e90c3f88ea66fee9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.606195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2589531b-75dd-4784-ac87-e7222b60748a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3157b4ceb3724e8e90c3f88ea66fee9a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:12.625338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1061/2104] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:12 2022] DELETE /identity/v3/users/3157b4ceb3724e8e90c3f88ea66fee9a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:12.915230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:12.916708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:12.929827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e3fbf-32a3-4353-a2da-dff1b1c9ccf9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:12.998118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:12.998791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:12.999070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:12.999346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] PATH_INFO: `/v3/projects/19faa3a42ef8456f869de1ebcf625944` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.000528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19faa3a42ef8456f869de1ebcf625944)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.002504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.035587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:13.061132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19faa3a42ef8456f869de1ebcf625944', 'request_id': 'req-e74b893a-99f6-4978-9aff-4db39460b1f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:13.068456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19faa3a42ef8456f869de1ebcf625944', 'request_id': 'req-e74b893a-99f6-4978-9aff-4db39460b1f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:13.075339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19faa3a42ef8456f869de1ebcf625944', 'request_id': 'req-e74b893a-99f6-4978-9aff-4db39460b1f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:13.088586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] The token cache is being invalidate because project 19faa3a42ef8456f869de1ebcf625944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:13.088985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:13.089895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e74b893a-99f6-4978-9aff-4db39460b1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:13.091614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1044/2105] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:12 2022] DELETE /identity/v3/projects/19faa3a42ef8456f869de1ebcf625944 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:13.096428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.096852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.097262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.129925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:13.131235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:13.252250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1062/2106] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:13.252683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.253557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.254092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.282464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:13.283876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:13.293787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1045/2107] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:13.298642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.309834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f10ea5-8201-416e-90a0-63bec5c6a933 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.315210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.315975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.316259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.317216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.318040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:13.318422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.322677 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.341258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1063/2108] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:13.342838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.354976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c53abe-35f6-44ea-a940-70bdb5822bb2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.362562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.363251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.363681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.364124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.364908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.367052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.411750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1046/2109] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:13.415893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.436063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1445fc2-faaa-4d22-9778-f5c14e92c048 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.512358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:13.537922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.540300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.540690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.542083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.542083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.544310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.574907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1064/2110] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:13.576197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.594413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b2917-9ea0-4b65-a2b5-98e63d5f0bec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.600063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.600894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.601306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.601764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.602561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.605446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.650571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1047/2111] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:13.655346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.675159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87730dde-b64c-479c-8368-529abbcd139b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.694850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.695645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.696159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.697725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.698908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.702878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdbb38-d911-4a9f-986b-2ce69cd12819 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.719271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1065/2112] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:13.723674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.733940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef818f3-2867-4f59-889a-d8f50c68545e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.742888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.743566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.743841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.744118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] PATH_INFO: `/v3/projects/24cd856e61e042d79d614e772b301ecd/users/5cc62aa52aba4875a0a8f52b75571140/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.770098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cd856e61e042d79d614e772b301ecd, user_id=5cc62aa52aba4875a0a8f52b75571140, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.773764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:13.788449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1048/2113] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:13 2022] PUT /identity/v3/projects/24cd856e61e042d79d614e772b301ecd/users/5cc62aa52aba4875a0a8f52b75571140/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:13.798720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.799006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.799295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.836174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] MFA Rules not processed for user `5cc62aa52aba4875a0a8f52b75571140`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:13.837376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:13.910712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1066/2114] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:13.916063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:13.983127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7627749-129c-49b2-9965-ba1d1040a143 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:13.988223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:13.989092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:13.989399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:13.989668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:13.990543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:13.993141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.026743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1049/2115] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:13 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:14.031991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.045025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954092ed-9400-4a9b-9787-7f0d265b136f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.129456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.130192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.130560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.130940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.132255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.134394 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.160376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1067/2116] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:14 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:14.164949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.175641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50784197-aa48-4459-8647-ad8965c92a10 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.181198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.181815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.182093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.182366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.183133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.184661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.197198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1050/2117] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:14.203425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.215881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fafd24-b37f-4412-ac56-fbd363e73749 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.224337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.224337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.224337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.224337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.225974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.225974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18674d-39d3-4fcc-8f82-e24ffd679c7a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.238076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1068/2118] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:14.242870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.261067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c6092-7a37-4f61-9db7-2c6a61fbebb1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.266678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.267339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.267638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.268021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] PATH_INFO: `/v3/projects/608655fda6df48b4973f8402c11e0bb1/users/d120e4e4ae5149668f4d87b6b5fd7857/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.284240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608655fda6df48b4973f8402c11e0bb1, user_id=d120e4e4ae5149668f4d87b6b5fd7857, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.287166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.300215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:14.305185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1051/2119] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:14 2022] PUT /identity/v3/projects/608655fda6df48b4973f8402c11e0bb1/users/d120e4e4ae5149668f4d87b6b5fd7857/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:14.315612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.316292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.316652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.352986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] MFA Rules not processed for user `d120e4e4ae5149668f4d87b6b5fd7857`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:14.354491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:14.377985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:14.440323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1069/2120] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:14.442277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.520343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb76a9-a46b-41ec-a73f-8de87252cb05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.525753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.526440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.526781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.527167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.527820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.529787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.562152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1052/2121] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:14 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:14.563448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.575418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5224-aa1f-4ed0-92b6-9e550c662cf7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.647616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.647836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.648066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.648343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.648988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.651232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.679658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1070/2122] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:14 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:14.680866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.691986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f242b7-1b6c-4907-80c2-bea15150757d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.701811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.702444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.702774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.703047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.703690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.705564 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.717796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1053/2123] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:14.722203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.732931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8af26-9ef2-418e-98ab-c35a171131be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.737662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.738318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.738610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.738931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] PATH_INFO: `/v3/projects/42eacbac721a42ae9c49f6d569e34010/users/003ce5b0a9674595a6baa87dd5c9d275/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.763617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eacbac721a42ae9c49f6d569e34010, user_id=003ce5b0a9674595a6baa87dd5c9d275, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.766573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.786933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1071/2124] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:14 2022] PUT /identity/v3/projects/42eacbac721a42ae9c49f6d569e34010/users/003ce5b0a9674595a6baa87dd5c9d275/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:14.791218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.802363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5ab95-2515-4a2c-8bb9-08e4a77be4ca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.879093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.879788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.880131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.880567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.881807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.883470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaba98-3960-4715-9cdd-d40da3bc032d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.893701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1054/2125] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:14.898405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:14.909056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bc413-7929-4332-80ce-b00aefe3654a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:14.913866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:14.914518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.914799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.915106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] PATH_INFO: `/v3/projects/42eacbac721a42ae9c49f6d569e34010/users/003ce5b0a9674595a6baa87dd5c9d275/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.917343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42eacbac721a42ae9c49f6d569e34010, user_id=003ce5b0a9674595a6baa87dd5c9d275, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:14.920918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:14.931073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1072/2126] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:14 2022] PUT /identity/v3/projects/42eacbac721a42ae9c49f6d569e34010/users/003ce5b0a9674595a6baa87dd5c9d275/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:14.941019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:14.941393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:14.941666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:14.975172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] MFA Rules not processed for user `003ce5b0a9674595a6baa87dd5c9d275`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:14.976370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:15.034920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:15.043977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1055/2127] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:15.056143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:15.057850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:15.116870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c871-e371-4766-9841-1cd76e956c6a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:15.120720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:15.121356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:15.121660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:15.122040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:15.149405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:15.181182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:15.182618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:15.191380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1073/2128] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:18.838784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:18.840212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:18.853072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85df98-4376-4f67-855b-eb32971bed67 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:18.857292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:18.858104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:18.858390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:18.859072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:18.926024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:18.927394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:18.936638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1056/2129] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:19.530531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:19.540750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a2bc2-c5d0-49ac-97f6-4c88800a07cf service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:19.544408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:19.545005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:19.545587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:19.545862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:19.556202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:19.557963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6c8ac-3a39-45c7-8eec-397435a6d030 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:19.568287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1074/2130] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:19.922781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:19.924605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.019454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4de2e-6089-4811-b14f-cd5696700515 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:20.024438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:20.029246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.029534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.029811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] PATH_INFO: `/v3/users/3b845fd64b904d71a41e054c3a2330d5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.031003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b845fd64b904d71a41e054c3a2330d5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:20.033157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:20.122698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b845fd64b904d71a41e054c3a2330d5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.132380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b845fd64b904d71a41e054c3a2330d5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.141843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b845fd64b904d71a41e054c3a2330d5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.142999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b845fd64b904d71a41e054c3a2330d5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.158462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b845fd64b904d71a41e054c3a2330d5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.165919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1057/2131] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:19 2022] DELETE /identity/v3/users/3b845fd64b904d71a41e054c3a2330d5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:20.170780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.171191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.171642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.192255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:20.193515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.264602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1075/2132] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:20.275125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:20.277475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.292369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:20.342395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c12e0-c89d-4718-bccb-4db48aa63b0f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:20.346191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:20.346797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.347079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.347351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.357287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:20.359923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a895-e9f5-4675-8a76-d9457f729ed4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:20.373315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1058/2133] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:20.646384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:20.649190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.676837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955df95-9530-45b1-aa46-af9d7e821939 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:20.687275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:20.687929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.688421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.688700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] PATH_INFO: `/v3/projects/0527e01480e44ca4b314cbca75204a0f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.690088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0527e01480e44ca4b314cbca75204a0f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:20.692033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:20.787548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0527e01480e44ca4b314cbca75204a0f', 'request_id': 'req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.799479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0527e01480e44ca4b314cbca75204a0f', 'request_id': 'req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.807183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0527e01480e44ca4b314cbca75204a0f', 'request_id': 'req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.819601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] The token cache is being invalidate because project 0527e01480e44ca4b314cbca75204a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:20.819971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.821290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f5c7f65-b7f7-4463-a4ef-cf2c32516947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:20.823324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1076/2134] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:20 2022] DELETE /identity/v3/projects/0527e01480e44ca4b314cbca75204a0f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:20.828215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.831510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.831510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.850039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:20.851249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.944786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1059/2135] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:20.955583 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:20.956008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:20.956481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:20.978747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:20.980039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:20.991815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1077/2136] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:20.996223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.012397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2633ae2-aacb-4f55-922c-6a62e742141f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.017657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.018253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.018628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.018955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.019835 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:21.019940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.029252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.039229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.043846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1060/2137] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:21.049069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7861-3048-4473-a4e2-978dc4f62c26 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.054005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.055158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.055520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.056070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.056561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.058675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.107995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1078/2138] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:21.113553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.139398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9803c93-7400-4abc-80f4-aec2382ab24e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.218690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.219543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.219974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.220451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.221288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.223596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.262552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1061/2139] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:21.263790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.283563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38323e68-81c1-4cbe-a1b2-de41956eeaf2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.294080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.294808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.295019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.295289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.297483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.306514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.314613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:21.333982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.336748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:21.341780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1079/2140] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:21.343693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.358783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf941-e55c-4045-beff-aee808d3f0cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.368526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a0c1a-2225-4124-b17f-71acfbf849b9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.369875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.370922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.371413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.372056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] PATH_INFO: `/v3/users/aafc4866f6a4414bb4a82cbe66f773b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.375259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafc4866f6a4414bb4a82cbe66f773b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.378197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.382387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.383281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.383628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.384478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.384886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.386711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.399733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1080/2141] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:21.404944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.422093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18e546-d631-4330-b032-13f33a165200 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.445477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.445583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:21.445662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.445662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.445662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] PATH_INFO: `/v3/projects/e67df8778b1a44338d4fbc215bae0d71/users/50ddd8fb390f4b378715d648ef758276/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.475750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafc4866f6a4414bb4a82cbe66f773b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:21.482549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67df8778b1a44338d4fbc215bae0d71, user_id=50ddd8fb390f4b378715d648ef758276, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.484921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafc4866f6a4414bb4a82cbe66f773b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:21.485582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60747454-859a-4cfd-8968-12d280e76cb7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.496980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafc4866f6a4414bb4a82cbe66f773b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:21.497831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafc4866f6a4414bb4a82cbe66f773b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:21.521366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1081/2142] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:21 2022] PUT /identity/v3/projects/e67df8778b1a44338d4fbc215bae0d71/users/50ddd8fb390f4b378715d648ef758276/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:21.527178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8161c844-159d-4670-9a7c-680a18be55b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafc4866f6a4414bb4a82cbe66f773b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:21.528290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.528579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.529769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.538355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1062/2143] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:21 2022] DELETE /identity/v3/users/aafc4866f6a4414bb4a82cbe66f773b6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:21.543013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.543296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.543570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.560312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] MFA Rules not processed for user `50ddd8fb390f4b378715d648ef758276`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:21.561694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:21.563091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:21.564258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:21.623031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1063/2144] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:21.627969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1082/2145] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:21.632406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.634000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:21.634175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.635543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:21.645588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d137d54-0136-412e-99de-a1aaafcb2ed6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.650555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.651248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.651434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.651756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.653090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.655024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.697577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726eca7-ef0b-4f0a-9a04-dadcad9b086a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.703072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1064/2146] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:21.703215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.703215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.703215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.703215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.704111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.714887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.716319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.725985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1083/2147] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:21.769637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178eb31-5ac3-4033-a35c-b9a19c96d503 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.776110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.776819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.778220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.778220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.778456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.780588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.814730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1065/2148] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:21.821222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.832602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f03d1-cbf0-4f8b-a9db-cb96ae7ad54a service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.838560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.839357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.839785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.840222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.840868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.842531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.855996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1084/2149] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:21.860154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.872610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b44b9-0795-445d-b0b0-72cad3b1513f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.877990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.878623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.878904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.879178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.879830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.881811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b16135-bb05-4dd5-9c9b-a8d899c105ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.895110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1066/2150] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:21.902411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:21.922930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab704426-e357-4b05-b523-f975cbd9228f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:21.928250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:21.928992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.929775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.930119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] PATH_INFO: `/v3/projects/6339d95ba17648b9ad0130c3de39ec4a/users/2bf88bfe147247a4a082e9e7eb0bc660/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.952793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6339d95ba17648b9ad0130c3de39ec4a, user_id=2bf88bfe147247a4a082e9e7eb0bc660, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:21.955512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:21.970895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1085/2151] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:21 2022] PUT /identity/v3/projects/6339d95ba17648b9ad0130c3de39ec4a/users/2bf88bfe147247a4a082e9e7eb0bc660/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:21.982687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:21.983214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:21.983214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:21.990581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:22.014008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] MFA Rules not processed for user `2bf88bfe147247a4a082e9e7eb0bc660`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:22.015264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.067355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.069104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.101020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1067/2152] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:22.105778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:22.106225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.155441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d4ebc-d657-409c-99c6-55f14f91c7a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.160248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.160874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.161179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.161454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] PATH_INFO: `/v3/projects/0a5e5a042d014387b27347c02f04e865` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.162649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5e5a042d014387b27347c02f04e865)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.164590 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.182700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75142e-e8db-439d-b537-f41649eec1a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.190457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.191182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.191746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.191746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.192342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.194325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.224574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5e5a042d014387b27347c02f04e865', 'request_id': 'req-2a720a4a-d202-4457-ab66-3c9870b69789'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.228581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1068/2153] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:22 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:22.232982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.235311 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5e5a042d014387b27347c02f04e865', 'request_id': 'req-2a720a4a-d202-4457-ab66-3c9870b69789'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.242070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5e5a042d014387b27347c02f04e865', 'request_id': 'req-2a720a4a-d202-4457-ab66-3c9870b69789'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.258510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] The token cache is being invalidate because project 0a5e5a042d014387b27347c02f04e865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:22.258812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.259513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.261474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1086/2154] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:22 2022] DELETE /identity/v3/projects/0a5e5a042d014387b27347c02f04e865 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:22.265959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.267864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.309156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc432d-42ca-4c45-ab2b-ed21c961f555 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.312622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.314056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.314056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.314056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.314659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.316785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.342309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1069/2155] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:22 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:22.346246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.347955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.353820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a720a4a-d202-4457-ab66-3c9870b69789 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.359135 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.359847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.361651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.361651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] PATH_INFO: `/v3/users/32323304a230402b95b5865ea8facf6c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.361651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32323304a230402b95b5865ea8facf6c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.362227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3940e59-0a7c-4516-aa71-d7a1df3e3695 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.363413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.367932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.368605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.368943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.369331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.370037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.371716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.373521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:22.390056 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1070/2156] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:22.394243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.406418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cf6e6-915d-4ec0-ae53-0b160a2b2001 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.412822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.413631 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.413973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.414287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] PATH_INFO: `/v3/projects/6b05fa73588f469caf05a0d037e546cc/users/d7faba1ea94645199d1edfb86c2dd104/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.434728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b05fa73588f469caf05a0d037e546cc, user_id=d7faba1ea94645199d1edfb86c2dd104, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.437758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.444714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32323304a230402b95b5865ea8facf6c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.446914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32323304a230402b95b5865ea8facf6c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.452829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:22.467254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32323304a230402b95b5865ea8facf6c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.471547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1071/2157] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:22 2022] PUT /identity/v3/projects/6b05fa73588f469caf05a0d037e546cc/users/d7faba1ea94645199d1edfb86c2dd104/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:22.472838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32323304a230402b95b5865ea8facf6c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.474132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.477262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.495540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32323304a230402b95b5865ea8facf6c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:22.526374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1087/2158] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:22 2022] DELETE /identity/v3/users/32323304a230402b95b5865ea8facf6c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:22.589861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce1450-487c-48c0-aa1b-67c97fca86b8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.672859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.673853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.674157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.674430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.675078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.677378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e179b-21e1-4023-abc1-9e0513466c52 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.692075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1072/2159] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:22.694321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.705147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffb234-0598-43b7-abd9-f8eb8d7e4253 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:22.717812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:22.720780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.721069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.722141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] PATH_INFO: `/v3/projects/6b05fa73588f469caf05a0d037e546cc/users/d7faba1ea94645199d1edfb86c2dd104/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.728004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b05fa73588f469caf05a0d037e546cc, user_id=d7faba1ea94645199d1edfb86c2dd104, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:22.733260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:22.748480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1088/2160] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:22 2022] PUT /identity/v3/projects/6b05fa73588f469caf05a0d037e546cc/users/d7faba1ea94645199d1edfb86c2dd104/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:22.761337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:22.761337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:22.761337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:22.816009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] MFA Rules not processed for user `d7faba1ea94645199d1edfb86c2dd104`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:22.816999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:22.918208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:22.925260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1073/2161] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:22.952369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:22.955737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.018376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60fad-4d19-451e-8bea-697ecc5805fa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:23.022763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:23.024029 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.024320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.024636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.028550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.098339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:23.099858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8131fc-5654-4287-9778-4be3a73d3e31 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:23.111779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1089/2162] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:23.362469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:23.364619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.521593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24700fd-a5b4-457d-ae5d-00684bea7b80 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:23.526097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:23.526933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.527383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.527806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] PATH_INFO: `/v3/projects/546eff2970f345eca27610b6418d0924` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.529096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] RBAC: Authorizing `identity:delete_project(project_id=546eff2970f345eca27610b6418d0924)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:23.531348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:23.599157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546eff2970f345eca27610b6418d0924', 'request_id': 'req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.607797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546eff2970f345eca27610b6418d0924', 'request_id': 'req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.615437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546eff2970f345eca27610b6418d0924', 'request_id': 'req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.631933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] The token cache is being invalidate because project 546eff2970f345eca27610b6418d0924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:23.632234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.633032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9583c7a0-dbc8-4e83-9af3-05c17f6f121e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.634973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1074/2163] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:23 2022] DELETE /identity/v3/projects/546eff2970f345eca27610b6418d0924 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:23.639813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.640325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.640748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.661988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:23.663455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.733804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1090/2164] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:23.741662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.742005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.742962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.748697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:23.750215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.772486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:23.774583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.775628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f50d07-136c-42f0-a925-a02373faa9b9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:23.782247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:23.782951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.783392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.783810 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] PATH_INFO: `/v3/users/5cc62aa52aba4875a0a8f52b75571140` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.785110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc62aa52aba4875a0a8f52b75571140)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:23.787205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:23.795068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:23.795515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1075/2165] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:23.796922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:23.812769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543d80-df63-48ae-a64a-8e1157523531 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:23.819906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:23.820633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.825245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.825632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.826842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:23.827839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:23.829578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:23.846623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1076/2166] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:23.851247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:23.866780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74fcac-34b8-4785-a2a9-ed31c62e368a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:23.873638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:23.874420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.874799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.875173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:23.876015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:23.878064 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc62aa52aba4875a0a8f52b75571140'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.878484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc62aa52aba4875a0a8f52b75571140'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.878598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:23.889031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc62aa52aba4875a0a8f52b75571140'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.889946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc62aa52aba4875a0a8f52b75571140'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.910415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3af985a9-d37f-464c-b37d-0cc831b7b4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc62aa52aba4875a0a8f52b75571140'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:23.948786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1077/2167] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:23 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:23.953040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:23.958374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:23.962218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1091/2168] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:23 2022] DELETE /identity/v3/users/5cc62aa52aba4875a0a8f52b75571140 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:23.966823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:23.971288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:23.971656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.011920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:24.014925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:24.073035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:24.098439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50958ad0-fe00-49ae-ab50-beac9fc93a6d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.153900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1092/2169] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:24.178609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.180294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:24.212904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.213742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.214094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.214437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.215150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.217308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.274711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1078/2170] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:23 2022] POST /identity/v3/users => generated 282 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:24.278683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.291422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2001f8a-ac63-4011-81c7-1028799dd241 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.299720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.300727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.301239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.301522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.302280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.302726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a4748e-5c4e-4e6f-8bb3-a2d43d4f5583 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.304645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.307269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.308084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.308462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.308846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.320411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1079/2171] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:24.320609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.321196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.321989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.332342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1093/2172] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:24.334334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92afebf-bb8b-4532-b32c-9c30245a983f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.339721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.340913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.340913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.340913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.341554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.343301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5cf03-c424-47e0-b9ba-9bb9449c30b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.367517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1080/2173] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:24.379594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.392611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66b8b-2023-4285-89c7-25c59292d04f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.398983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.399719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.400200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.400635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] PATH_INFO: `/v3/projects/912fa846b51647678e981950e8741d9a/users/061052054914431db212078ee32853cc/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.429865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=912fa846b51647678e981950e8741d9a, user_id=061052054914431db212078ee32853cc, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.436646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.469174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1094/2174] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:24 2022] PUT /identity/v3/projects/912fa846b51647678e981950e8741d9a/users/061052054914431db212078ee32853cc/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:24.472841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.473142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.473429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.502882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] MFA Rules not processed for user `061052054914431db212078ee32853cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:24.504126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:24.601721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1081/2175] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:24.605975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.616570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:24.707291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7037a-8ebe-4713-9d18-ae892c0adcbd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.729714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.730487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.730832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.731234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.732318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.735872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.776777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1095/2176] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:24 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:24.776929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.776929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:24.778378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.897721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4af2ce-62f5-4dbf-8649-984c5cad1860 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.909220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7b174-7b2e-4d08-aec8-539a5fd3b875 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.909568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.909568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.909568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.909568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.910205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.917179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:24.917381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:24.952901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1096/2177] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:24 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:24.954731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:24.957925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:24.975352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5ed68-1fdf-4812-8542-8854aa0248b4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:24.981438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:24.982370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:24.982977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:24.983434 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] PATH_INFO: `/v3/projects/24cd856e61e042d79d614e772b301ecd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:24.984991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24cd856e61e042d79d614e772b301ecd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:24.987147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.002278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.003005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.003372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.003738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.005664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:25.050670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24cd856e61e042d79d614e772b301ecd', 'request_id': 'req-4646d831-1dfa-4107-803a-596a8157b096'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.064512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24cd856e61e042d79d614e772b301ecd', 'request_id': 'req-4646d831-1dfa-4107-803a-596a8157b096'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.078368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24cd856e61e042d79d614e772b301ecd', 'request_id': 'req-4646d831-1dfa-4107-803a-596a8157b096'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.084663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.086667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.090370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:25.096562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] The token cache is being invalidate because project 24cd856e61e042d79d614e772b301ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:25.097070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.098650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.100250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1082/2178] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:25.102815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.113771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1097/2179] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:24 2022] DELETE /identity/v3/projects/24cd856e61e042d79d614e772b301ecd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:25.118643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.122647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:25.124103 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7acfa47-30a5-4821-8254-39a6c7759f69 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:25.235118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646d831-1dfa-4107-803a-596a8157b096 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:25.240944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.241247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.242721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.243116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.243493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] PATH_INFO: `/v3/users/d120e4e4ae5149668f4d87b6b5fd7857` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.243659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.244225 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.244493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.245171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.245541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d120e4e4ae5149668f4d87b6b5fd7857)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.246778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.248341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.275092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1083/2180] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:25.278064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.279854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:25.338133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa3b03-a544-424e-b0b0-5ce21ea82fe4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:25.349346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.350496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.353239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.353760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] PATH_INFO: `/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74/users/04e25b8b85bc46cd99890fc1869a3962/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.361553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd120e4e4ae5149668f4d87b6b5fd7857'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.362742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd120e4e4ae5149668f4d87b6b5fd7857'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.377649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd120e4e4ae5149668f4d87b6b5fd7857'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.378543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd120e4e4ae5149668f4d87b6b5fd7857'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.390151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd120e4e4ae5149668f4d87b6b5fd7857'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:25.397261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ffaebc3a1f499e91a32b15ff45ea74, user_id=04e25b8b85bc46cd99890fc1869a3962, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.401870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.427365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1084/2181] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:25 2022] PUT /identity/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74/users/04e25b8b85bc46cd99890fc1869a3962/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:25.432269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.440723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1098/2182] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:25 2022] DELETE /identity/v3/users/d120e4e4ae5149668f4d87b6b5fd7857 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:25.453757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:25.585210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:25.662055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6845f8-fc04-41fa-9c17-a47ca76e008b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:25.809066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.809792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.812454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.813683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.814689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.817170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267c433-e685-484c-baf8-5073112c22d0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.847183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1085/2183] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:25.849287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.867798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c67d6f-679a-466c-8011-87d76ab21125 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:25.876050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:25.877049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.877564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.878039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] PATH_INFO: `/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74/users/04e25b8b85bc46cd99890fc1869a3962/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.882255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ffaebc3a1f499e91a32b15ff45ea74, user_id=04e25b8b85bc46cd99890fc1869a3962, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:25.886848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:25.916427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1099/2184] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:25 2022] PUT /identity/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74/users/04e25b8b85bc46cd99890fc1869a3962/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:25.926818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:25.927158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:25.927443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:25.972129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:25.974885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:25.997584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] MFA Rules not processed for user `04e25b8b85bc46cd99890fc1869a3962`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:25.997584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.144296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1086/2185] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:26.150436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac5e0-9ce2-4e3a-b6ba-f063279fcdf1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:26.164783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:26.168623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.184409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:26.185068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:26.185392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:26.185668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] PATH_INFO: `/v3/projects/608655fda6df48b4973f8402c11e0bb1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:26.188400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=608655fda6df48b4973f8402c11e0bb1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:26.193421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:26.208381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:26.322509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08dea1-5928-4de0-b134-bd3a6a1bbb18 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:26.338049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608655fda6df48b4973f8402c11e0bb1', 'request_id': 'req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.343425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:26.349626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608655fda6df48b4973f8402c11e0bb1', 'request_id': 'req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.356167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608655fda6df48b4973f8402c11e0bb1', 'request_id': 'req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.369267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] The token cache is being invalidate because project 608655fda6df48b4973f8402c11e0bb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:26.369558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.370225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.371727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1100/2186] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:25 2022] DELETE /identity/v3/projects/608655fda6df48b4973f8402c11e0bb1 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:26.372490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:26.373795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.389090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:26.389919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:26.390263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:26.390586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:26.392367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.442854 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef0fa-bc09-4f0c-a1e1-a8a11233c9ee admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:26.447796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:26.448662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:26.448985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:26.449458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:26.451853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.461990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:26.463481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:26.478392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1087/2187] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:26.479139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:26.480459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.551517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:26.553623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:26.566237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1101/2188] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:26.596334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:26.598014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.598420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b368c-44ce-43bb-950d-5b13e4e374a2 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:26.605201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:26.606086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:26.606593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:26.607051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] PATH_INFO: `/v3/users/003ce5b0a9674595a6baa87dd5c9d275` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:26.608518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=003ce5b0a9674595a6baa87dd5c9d275)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:26.610590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:26.614767 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720209f7-e525-43fe-a205-c6456d7c52cc service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:26.623257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:26.624013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:26.624292 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:26.624522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:26.631956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:26.695991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003ce5b0a9674595a6baa87dd5c9d275'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.704275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003ce5b0a9674595a6baa87dd5c9d275'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.715016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003ce5b0a9674595a6baa87dd5c9d275'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.716588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003ce5b0a9674595a6baa87dd5c9d275'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.730232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:26.740436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003ce5b0a9674595a6baa87dd5c9d275'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:26.750638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:26.750832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1088/2189] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:26 2022] DELETE /identity/v3/users/003ce5b0a9674595a6baa87dd5c9d275 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:26.790115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:26.791667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:26.802181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1102/2190] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:27.311449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.315063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:27.425386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae3f3c-428d-4ada-beed-8d20e2326e7a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.431277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.433705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.434961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.435071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:27.435245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.435522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] PATH_INFO: `/v3/projects/42eacbac721a42ae9c49f6d569e34010` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.436705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42eacbac721a42ae9c49f6d569e34010)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.447544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.453871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f6e28-a61f-43cd-abb5-8bc895beb5bd service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.459280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.460055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.460456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.460890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] PATH_INFO: `/v3/users/561fa6572e0b4b949d1c09ee9e9d5e66` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.462214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561fa6572e0b4b949d1c09ee9e9d5e66)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.464938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.521163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42eacbac721a42ae9c49f6d569e34010', 'request_id': 'req-d188950c-acae-4b05-9dcd-e1df7523e983'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.529243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42eacbac721a42ae9c49f6d569e34010', 'request_id': 'req-d188950c-acae-4b05-9dcd-e1df7523e983'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.536848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42eacbac721a42ae9c49f6d569e34010', 'request_id': 'req-d188950c-acae-4b05-9dcd-e1df7523e983'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.540363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561fa6572e0b4b949d1c09ee9e9d5e66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.540493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561fa6572e0b4b949d1c09ee9e9d5e66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.548809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561fa6572e0b4b949d1c09ee9e9d5e66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.549493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561fa6572e0b4b949d1c09ee9e9d5e66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.552403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] The token cache is being invalidate because project 42eacbac721a42ae9c49f6d569e34010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:27.552732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.553611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d188950c-acae-4b05-9dcd-e1df7523e983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.555962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1089/2191] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:27 2022] DELETE /identity/v3/projects/42eacbac721a42ae9c49f6d569e34010 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:27.559382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-08adfd27-4b70-4b3b-b98e-38b1e22cf401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561fa6572e0b4b949d1c09ee9e9d5e66'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:27.560841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.561348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.561858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.585845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1103/2192] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:27 2022] DELETE /identity/v3/users/561fa6572e0b4b949d1c09ee9e9d5e66 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:27.586195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:27.587994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:27.667282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1090/2193] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:27.674921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.675212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.675490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.696814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:27.698231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:27.706223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1104/2194] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:27.710386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.720336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458e4eb-c9e3-4afc-a118-660e87072d40 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.727964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.728563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.728841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.729129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.729789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:27.730124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.731709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.746934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1091/2195] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:27.748855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.760053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b656add-dfd2-4128-9275-9cb4676bd33d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.768490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.769246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.769597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.769901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.770514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.772531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.828392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1105/2196] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:27 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:27.834346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.844520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.847109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:27.852431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d6e4a-c869-4ae7-9a85-c8b296e96e5c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.858355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:27.934916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.935529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.935825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.936100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.936731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.940361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.945859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f010-de77-453e-aa01-51d65dcd4e57 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.953943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.954791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.955166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.955562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] PATH_INFO: `/v3/projects/78f0be015ba44cdfa6c64038e16bd5cb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.956790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f0be015ba44cdfa6c64038e16bd5cb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:27.959059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:27.966696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1092/2197] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:27 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:27.972974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:27.987542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18ad28-6077-4100-b829-b0089d80507c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:27.997046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:27.998019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:27.999264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:27.999264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:27.999586 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.001837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.009807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:28.027930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f0be015ba44cdfa6c64038e16bd5cb', 'request_id': 'req-760d8149-7444-419c-ae9c-546cc7cb9546'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:28.033365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1093/2198] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:28.034801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.038663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f0be015ba44cdfa6c64038e16bd5cb', 'request_id': 'req-760d8149-7444-419c-ae9c-546cc7cb9546'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:28.046652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f0be015ba44cdfa6c64038e16bd5cb', 'request_id': 'req-760d8149-7444-419c-ae9c-546cc7cb9546'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:28.052983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046d1ff-5203-4db1-9b56-0493d1ee3112 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.063422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] The token cache is being invalidate because project 78f0be015ba44cdfa6c64038e16bd5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:28.063780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:28.064773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-760d8149-7444-419c-ae9c-546cc7cb9546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:28.066886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1106/2199] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:27 2022] DELETE /identity/v3/projects/78f0be015ba44cdfa6c64038e16bd5cb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:28.072181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df3c9-828f-4224-af0f-d034d2f845ce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.072462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df3c9-828f-4224-af0f-d034d2f845ce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.072733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df3c9-828f-4224-af0f-d034d2f845ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.095827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0e6df3c9-828f-4224-af0f-d034d2f845ce None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:28.097470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e6df3c9-828f-4224-af0f-d034d2f845ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.198105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.198906 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.199252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.199597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.200327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.202068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.221912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1094/2200] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:28.227221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.229617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.249240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223d4c7-3add-4ca7-b11b-f802195b388a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.253172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1107/2201] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:28.255044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.255903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.256285 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.257315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] PATH_INFO: `/v3/projects/7d030bb54a7c4e489734b7c86b40b464/users/1cc519da0b5d40268b29bb1bef89049b/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.258398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.258675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.258950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.274986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d030bb54a7c4e489734b7c86b40b464, user_id=1cc519da0b5d40268b29bb1bef89049b, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.278058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f273e44-25d9-46a8-bbe1-79bb0168c12d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.279633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:28.280828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.293346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1108/2202] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:28.297332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.297673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1095/2203] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:28 2022] PUT /identity/v3/projects/7d030bb54a7c4e489734b7c86b40b464/users/1cc519da0b5d40268b29bb1bef89049b/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:28.305194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.305810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.306088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.311109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09d92-1eec-436a-aef2-deb90954bb96 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.328344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:28.329820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.389600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.390223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.390552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.390826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.391498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:28.391802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.394014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.400637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:28.408150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1109/2204] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:28.412504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.415869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1096/2205] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:28.425905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.427328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.433817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb769db-00ac-4a71-8b49-a29198c82506 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.439822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.440558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.440848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.441159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.441779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.444077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.480666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1110/2206] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:28.486192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.498521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb75f8-7238-42ca-81ad-370b70b40a45 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.519974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:28.571148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.571663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bcc0a-71f0-41a6-97bd-af957f5b1f5c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.572192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.572715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.573242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.577631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.578353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.578727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.579101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.579860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.582346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.588899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.590390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.611529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1111/2207] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:28.612510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.615352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1097/2208] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:28.626390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57650927-ef0f-4dec-b44f-1dc1ff67bc58 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.631432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.632433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.633397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.633567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.634216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.635930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.646480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1112/2209] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:28.650587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.660428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d08-642c-4158-851d-e8f891d06edc service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.665916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.666946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.666946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.667510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.668251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.669991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995d27-36f3-440d-993e-d0b6a1730de6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.682345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1098/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:28.686519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.696999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928b076-8f59-4215-bce5-b4af49fac4db admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.701736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.702382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.702656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.702930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] PATH_INFO: `/v3/projects/57f426d9b88f4fd98f8c0226209aa018/users/c6f91a084954447f816ca7b789536ef3/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.718167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f426d9b88f4fd98f8c0226209aa018, user_id=c6f91a084954447f816ca7b789536ef3, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.721523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.735295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1113/2211] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:28 2022] PUT /identity/v3/projects/57f426d9b88f4fd98f8c0226209aa018/users/c6f91a084954447f816ca7b789536ef3/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:28.745944 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.746232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.746503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.784638 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] MFA Rules not processed for user `c6f91a084954447f816ca7b789536ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:28.785918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.866197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1099/2212] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:28.879502 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:28.881526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:28.958986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d13bb-eb1d-4a5a-85b6-9aeec87559f3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:28.965318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:28.965971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:28.966219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:28.966541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:28.974712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:28.976290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e7800-9043-43d4-923f-937f9b348e4a service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:28.985065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1114/2213] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:29.278903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:29.327226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:29.369089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18912e25-f544-4a10-a5f5-9043c42e2ee7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:29.374851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:29.375657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:29.375997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:29.376339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] PATH_INFO: `/v3/users/c6f91a084954447f816ca7b789536ef3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:29.377539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f91a084954447f816ca7b789536ef3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:29.380183 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:29.444821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f91a084954447f816ca7b789536ef3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:29.457159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f91a084954447f816ca7b789536ef3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:29.475590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f91a084954447f816ca7b789536ef3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:29.477843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f91a084954447f816ca7b789536ef3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:29.497651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a33aee55-6638-497f-8566-e2d59dbd61a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f91a084954447f816ca7b789536ef3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:29.511650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1100/2214] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:29 2022] DELETE /identity/v3/users/c6f91a084954447f816ca7b789536ef3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:29.516181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:29.516727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:29.516727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:29.517657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:29.518016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:29.518410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:29.539457 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:29.540668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:29.556375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] MFA Rules not processed for user `1cc519da0b5d40268b29bb1bef89049b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:29.558395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:29.611236 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1115/2215] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:29.624350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:29.625942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:29.636263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:29.657607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1101/2216] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:29.686701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:29.689205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:29.725205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986aeb6-6c44-4606-b451-46435cacac16 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:29.729652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:29.730659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:29.731668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:29.732131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:29.744018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:29.746113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:29.749888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:29.760689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1116/2217] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:29.778616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd03f7-4f92-440c-97e2-ebff77532a91 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:29.783055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:29.783819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:29.784115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:29.784396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:29.797994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:29.799935 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687cd52-33a7-4412-a621-1ad02f3f2fff service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:29.812993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1102/2218] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:30.143373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.145865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.186065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da9e95-2268-419d-b29c-c3df397495e9 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.198083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.198738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.199094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.202874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] PATH_INFO: `/v3/projects/57f426d9b88f4fd98f8c0226209aa018` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.205351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f426d9b88f4fd98f8c0226209aa018)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.209139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.333252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f426d9b88f4fd98f8c0226209aa018', 'request_id': 'req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.355484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f426d9b88f4fd98f8c0226209aa018', 'request_id': 'req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.375675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f426d9b88f4fd98f8c0226209aa018', 'request_id': 'req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.401069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] The token cache is being invalidate because project 57f426d9b88f4fd98f8c0226209aa018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:30.401069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.404760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02229bba-45ca-4a7d-be5a-9bcecf49c6b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.411476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.411779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.412047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.415118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1117/2219] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:30 2022] DELETE /identity/v3/projects/57f426d9b88f4fd98f8c0226209aa018 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:30.433431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:30.434647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.528101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1103/2220] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:30.538894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.539378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.539378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.562312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:30.563719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.577196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.577811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1118/2221] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:30.597305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e018b9b-983e-41f4-bdbe-e8cee158aa27 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.597305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.597305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.597305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.597687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.597687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:30.597687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.599304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.616135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1104/2222] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:15:30.616990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.628823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3545cde-0695-4e66-9fb3-682351a12400 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.634741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.635342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.635651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.635924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.636559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.638647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.667389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1119/2223] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:30 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:30.671578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.681990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bedbd4-c003-4843-8f5d-8cab663e7cc7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.759408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.760149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.760523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.760981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.762003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.764592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.779751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.782093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.795513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b14e11-c8b4-401c-9ae2-aaacd707926f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.801393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.802033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.802901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.803203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] PATH_INFO: `/v3/users/50ddd8fb390f4b378715d648ef758276` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.805013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ddd8fb390f4b378715d648ef758276)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.806986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.815997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1105/2224] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:30 2022] POST /identity/v3/users => generated 283 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:30.821673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.839554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b10cb-27b3-4ccc-b9ca-4188bcc88965 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.844982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.845734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.846121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.846464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.851003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.852799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.868326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1106/2225] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:30.869684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ddd8fb390f4b378715d648ef758276'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.870326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.870544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ddd8fb390f4b378715d648ef758276'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.879429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ddd8fb390f4b378715d648ef758276'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.880164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ddd8fb390f4b378715d648ef758276'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.880556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63402655-b0bb-4a48-a75b-0d58cbe5c0b1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.884138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:30.887358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.888045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.888344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.888643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.889384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:30.891096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:30.891290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f7492b70-80b5-4646-9e9c-0d95d3fbb1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ddd8fb390f4b378715d648ef758276'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:30.902607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1107/2226] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:30.907410 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:30.910152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.917988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1120/2227] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:30 2022] DELETE /identity/v3/users/50ddd8fb390f4b378715d648ef758276 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:30.922335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.922625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.922902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:30.926893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b47a-5514-4eab-bd3c-5fdc97544a5b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:30.944603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:30.946596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:30.972993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:30.991258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:30.992457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:30.992861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:30.993184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] PATH_INFO: `/v3/projects/6af37c4fe3a3410589dba985b7449711/users/5b2bca395e494d78abb69ef4ba04fe1b/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.008413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af37c4fe3a3410589dba985b7449711, user_id=5b2bca395e494d78abb69ef4ba04fe1b, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.009496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1121/2228] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:31.011139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca87b5-683b-4423-a59a-e4cc1f6b68aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.019866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.021559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.026064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1108/2229] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:30 2022] PUT /identity/v3/projects/6af37c4fe3a3410589dba985b7449711/users/5b2bca395e494d78abb69ef4ba04fe1b/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:31.036428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.036662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.036941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.064444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] MFA Rules not processed for user `5b2bca395e494d78abb69ef4ba04fe1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:31.065584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.078849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57787ecc-0834-4af1-97bf-04813e1a3436 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.131406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1109/2230] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:31.135718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.135718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.135718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.135718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.136393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.180017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:31.199097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.200482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0621-fd37-44aa-a3df-8074ed68c115 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.200603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.205017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.205680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.206409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.206409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.206833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.208755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0a2ac-e872-4897-b3cd-c702baaa1184 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.210712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1122/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:15:31.239261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1110/2232] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:31 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:31.244156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.254726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677d052-bdab-4f3f-bb21-00ce6ab9a884 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.356313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.356549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.356842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.359855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.360802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.361419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.361911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.363230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.365482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.384908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] MFA Rules not processed for user `1cc519da0b5d40268b29bb1bef89049b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:31.386259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.399936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1123/2233] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:31 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:31.401710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.415596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ae58e-b98f-4886-b03b-16e2e5994eb9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.430370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.431154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.431432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.431702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.432353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.434132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.444726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1124/2234] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:31.448979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.456713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:31.461893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1aa44-80cd-4d9b-9a8a-5d13ca620d4e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.484644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.485515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.485908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.486227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] PATH_INFO: `/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0/users/b22b218a9c96482c8fbff6384c16effb/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.490403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1111/2235] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:31.504137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa5b0d9df5f4021af714fe57ac5e0d0, user_id=b22b218a9c96482c8fbff6384c16effb, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.507629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.526794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1125/2236] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:31 2022] PUT /identity/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0/users/b22b218a9c96482c8fbff6384c16effb/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:31.530766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.546602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e81d5d-3ce4-4f98-9376-516f743f45a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.565737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.567923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.663295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d5287-bf02-40e7-bd2f-abbf46eff707 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.663472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.665272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.667960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.668260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.668733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.669054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.669392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.669667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.670014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] PATH_INFO: `/v3/projects/e67df8778b1a44338d4fbc215bae0d71` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.671039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67df8778b1a44338d4fbc215bae0d71)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.671767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.673224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.689619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1112/2237] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:31.694677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.713310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc69619-3d34-4853-8d04-754b99e7a0fc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.719943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:31.735553 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67df8778b1a44338d4fbc215bae0d71', 'request_id': 'req-4de5a5b3-b604-4d4e-a341-a71c47f9c005'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.746367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67df8778b1a44338d4fbc215bae0d71', 'request_id': 'req-4de5a5b3-b604-4d4e-a341-a71c47f9c005'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.752872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67df8778b1a44338d4fbc215bae0d71', 'request_id': 'req-4de5a5b3-b604-4d4e-a341-a71c47f9c005'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.771637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] The token cache is being invalidate because project e67df8778b1a44338d4fbc215bae0d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:31.771928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.772600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.774689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1126/2238] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:31 2022] DELETE /identity/v3/projects/e67df8778b1a44338d4fbc215bae0d71 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:31.781862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.783256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.811359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.812024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.812304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.812575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] PATH_INFO: `/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0/users/b22b218a9c96482c8fbff6384c16effb/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.814767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa5b0d9df5f4021af714fe57ac5e0d0, user_id=b22b218a9c96482c8fbff6384c16effb, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.817473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3aa450-603f-463f-81a9-43d7f34744cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.829575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1113/2239] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:31 2022] PUT /identity/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0/users/b22b218a9c96482c8fbff6384c16effb/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:31.840162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.840491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.840764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.852833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5a5b3-b604-4d4e-a341-a71c47f9c005 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:31.870365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] MFA Rules not processed for user `b22b218a9c96482c8fbff6384c16effb`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:31.871612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.920878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:31.921662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:31.921948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:31.922451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] PATH_INFO: `/v3/users/2bf88bfe147247a4a082e9e7eb0bc660` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:31.923652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf88bfe147247a4a082e9e7eb0bc660)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:31.925750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:31.946990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:31.956422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1114/2240] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:31.965580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:31.967085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:31.987425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf88bfe147247a4a082e9e7eb0bc660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.988373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf88bfe147247a4a082e9e7eb0bc660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.996571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf88bfe147247a4a082e9e7eb0bc660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:31.997415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf88bfe147247a4a082e9e7eb0bc660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.007006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf88bfe147247a4a082e9e7eb0bc660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.016738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:32.033460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1127/2241] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:31 2022] DELETE /identity/v3/users/2bf88bfe147247a4a082e9e7eb0bc660 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:32.035847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c557cb5-2435-44bf-99b2-baffbc268f14 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:32.091323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:32.091556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:32.091843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:32.092108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:32.094571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:32.178758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:32.181705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:32.203472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1115/2242] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:32.327321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:32.328597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:32.342612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859572a-6fa3-452b-9842-272f583e2459 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:32.347264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:32.347945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:32.348308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:32.348659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:32.352259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:32.405348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:32.406895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:32.431999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:32.433596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:32.451768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1128/2243] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:32.500849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaa803-56bf-4376-b57e-5c655394befa service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:32.509399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:32.510139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:32.510439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:32.510805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] PATH_INFO: `/v3/projects/6339d95ba17648b9ad0130c3de39ec4a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:32.512016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6339d95ba17648b9ad0130c3de39ec4a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:32.514403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:32.525958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:32.607012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6339d95ba17648b9ad0130c3de39ec4a', 'request_id': 'req-19f790a7-0593-45d1-b44e-90efe2b73b08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.616031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6339d95ba17648b9ad0130c3de39ec4a', 'request_id': 'req-19f790a7-0593-45d1-b44e-90efe2b73b08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.635297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6339d95ba17648b9ad0130c3de39ec4a', 'request_id': 'req-19f790a7-0593-45d1-b44e-90efe2b73b08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.658393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] The token cache is being invalidate because project 6339d95ba17648b9ad0130c3de39ec4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:32.658966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.659739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.665954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:32.667482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1116/2244] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:32 2022] DELETE /identity/v3/projects/6339d95ba17648b9ad0130c3de39ec4a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:32.677238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81958ab3-4c3c-4b37-84a8-ef5b0e929321 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:32.739657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:32.740285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:32.740560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:32.740831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] PATH_INFO: `/v3/users/d7faba1ea94645199d1edfb86c2dd104` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:32.742554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7faba1ea94645199d1edfb86c2dd104)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:32.744467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:32.794814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7faba1ea94645199d1edfb86c2dd104'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.795468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7faba1ea94645199d1edfb86c2dd104'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.802646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7faba1ea94645199d1edfb86c2dd104'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.803284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7faba1ea94645199d1edfb86c2dd104'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.811811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4f1e5b80-990f-44e5-8230-6cc1010e7b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7faba1ea94645199d1edfb86c2dd104'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:32.829494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1129/2245] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:32 2022] DELETE /identity/v3/users/d7faba1ea94645199d1edfb86c2dd104 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:33.092700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.094176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:33.107702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f790a7-0593-45d1-b44e-90efe2b73b08 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.168576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.169231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.169508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.169775 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] PATH_INFO: `/v3/projects/6b05fa73588f469caf05a0d037e546cc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.170881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b05fa73588f469caf05a0d037e546cc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.172792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.218773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b05fa73588f469caf05a0d037e546cc', 'request_id': 'req-38ba73db-4f17-4dce-bf6d-f951b46bcf11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:33.224877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b05fa73588f469caf05a0d037e546cc', 'request_id': 'req-38ba73db-4f17-4dce-bf6d-f951b46bcf11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:33.231091 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b05fa73588f469caf05a0d037e546cc', 'request_id': 'req-38ba73db-4f17-4dce-bf6d-f951b46bcf11'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:33.243191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] The token cache is being invalidate because project 6b05fa73588f469caf05a0d037e546cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:33.243482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:33.244094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-38ba73db-4f17-4dce-bf6d-f951b46bcf11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:33.245587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1117/2246] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:33 2022] DELETE /identity/v3/projects/6b05fa73588f469caf05a0d037e546cc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:33.250463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.250746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.251016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.270046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:33.271230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:33.276032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:33.341415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1130/2247] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:33.345877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.346184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.346485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.366825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:33.368036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:33.375875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1118/2248] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:33.379942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.390252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c705d-f27e-4d5f-87d9-c47d7e43a09e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.394749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.395337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.395617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.395887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.396521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:33.396816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.399016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.410180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1131/2249] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:33.414338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.423904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a5904-ad13-4f77-81fb-415183117e1e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.428625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.429263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.429539 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.429808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.430475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.432375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.452892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:33.462388 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1119/2250] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:33 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:33.466153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.476055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667c1e-62ae-41e3-870c-b1d27c57d3a3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.531349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.531945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.532230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.532505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.533156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.535103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.554418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1132/2251] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:33 2022] POST /identity/v3/users => generated 296 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:33.558547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.568887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f40b7-84c7-4c17-b314-1b1213e08861 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.574043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.574652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.574934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.575209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.575857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.577508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.588535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1120/2252] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:33.593208 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.603419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd161cd7-964f-4566-9e12-03a9a22db664 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.608008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.608602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.608883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.609199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.609831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.611422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d31f-f796-432a-b700-91589adc91f4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.622939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1133/2253] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:33.627307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.638397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a7bd3-280a-4f35-9316-100fd338d8f2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.643004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.643643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.643923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.644198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] PATH_INFO: `/v3/projects/f71bcba137f947e28bd7b8c0c8388ea7/users/41df4be7d90e4f2b8f2147489ea6e2e7/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.658953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71bcba137f947e28bd7b8c0c8388ea7, user_id=41df4be7d90e4f2b8f2147489ea6e2e7, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.661647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.675061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1121/2254] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:33 2022] PUT /identity/v3/projects/f71bcba137f947e28bd7b8c0c8388ea7/users/41df4be7d90e4f2b8f2147489ea6e2e7/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:33.685441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.685910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.686006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.715039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] MFA Rules not processed for user `41df4be7d90e4f2b8f2147489ea6e2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:33.716241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:33.784409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1134/2255] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:33.798257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:33.800282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:33.880732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eccd1d-0655-4da1-a953-a5d6de3043c1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:33.885202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:33.885816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:33.886095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:33.886368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:33.895269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:33.897970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590a927-fae5-4ffd-884f-ee051731b8f4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:33.906132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1122/2256] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:36.457890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:36.459291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:36.466085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:36.524510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f70ad3-fe13-4660-b79d-10dc5def3e14 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:36.528830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:36.529478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:36.529757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:36.530070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] PATH_INFO: `/v3/users/5b2bca395e494d78abb69ef4ba04fe1b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:36.531186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2bca395e494d78abb69ef4ba04fe1b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:36.533088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:36.576666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2bca395e494d78abb69ef4ba04fe1b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:36.577757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2bca395e494d78abb69ef4ba04fe1b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:36.584619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2bca395e494d78abb69ef4ba04fe1b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:36.585222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2bca395e494d78abb69ef4ba04fe1b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:36.593444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2bca395e494d78abb69ef4ba04fe1b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:36.610692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1135/2257] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:36 2022] DELETE /identity/v3/users/5b2bca395e494d78abb69ef4ba04fe1b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:36.615111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:36.615560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:36.615649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:36.635613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:36.636825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:36.695281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1123/2258] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:36.705375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:36.706769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:36.764231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f295f0d-fe02-4880-8aa3-927afc4624a5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:36.767858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:36.768463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:36.768744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:36.769541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:36.778034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:36.779402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:36.788607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1136/2259] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:37.050116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:37.051593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:37.064289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67065013-3157-4b13-ab1b-c8e5db13e4ff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:37.068863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:37.069520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:37.069799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:37.070075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] PATH_INFO: `/v3/projects/6af37c4fe3a3410589dba985b7449711` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:37.071772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af37c4fe3a3410589dba985b7449711)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:37.073764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:37.122832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af37c4fe3a3410589dba985b7449711', 'request_id': 'req-26c04498-bb9a-473a-ae07-1ec51f33a3bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.129600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af37c4fe3a3410589dba985b7449711', 'request_id': 'req-26c04498-bb9a-473a-ae07-1ec51f33a3bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.135661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:37.137074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af37c4fe3a3410589dba985b7449711', 'request_id': 'req-26c04498-bb9a-473a-ae07-1ec51f33a3bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.152372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] The token cache is being invalidate because project 6af37c4fe3a3410589dba985b7449711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:37.152662 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.153316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.154812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1124/2260] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:37 2022] DELETE /identity/v3/projects/6af37c4fe3a3410589dba985b7449711 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:37.159043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:37.169307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a9e62-8ae5-43f7-8b21-67a04617b6c2 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:37.194171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:37.227935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:37.228557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:37.228831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:37.229100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] PATH_INFO: `/v3/users/b22b218a9c96482c8fbff6384c16effb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:37.230244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22b218a9c96482c8fbff6384c16effb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:37.232156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:37.279113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22b218a9c96482c8fbff6384c16effb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.279753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22b218a9c96482c8fbff6384c16effb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.286387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22b218a9c96482c8fbff6384c16effb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.287002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22b218a9c96482c8fbff6384c16effb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.294161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-112ea00d-7c3a-4503-8573-d3bda941fc14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22b218a9c96482c8fbff6384c16effb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.311985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1137/2261] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:37 2022] DELETE /identity/v3/users/b22b218a9c96482c8fbff6384c16effb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:37.724016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:37.725641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:37.748845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c04498-bb9a-473a-ae07-1ec51f33a3bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:37.818731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:37.819358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:37.819633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:37.819905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] PATH_INFO: `/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:37.821045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa5b0d9df5f4021af714fe57ac5e0d0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:37.823108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:37.867687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa5b0d9df5f4021af714fe57ac5e0d0', 'request_id': 'req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.873610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa5b0d9df5f4021af714fe57ac5e0d0', 'request_id': 'req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.879148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa5b0d9df5f4021af714fe57ac5e0d0', 'request_id': 'req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.891801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] The token cache is being invalidate because project baa5b0d9df5f4021af714fe57ac5e0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:37.892087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.892739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db92b7cb-bcb6-4f91-82f6-367db8ee59e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:37.894251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1125/2262] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:37 2022] DELETE /identity/v3/projects/baa5b0d9df5f4021af714fe57ac5e0d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:37.901946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:37.902427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:37.902545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.043737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:38.044963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:38.102381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1138/2263] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:38.110088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.110375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.110654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.129928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:38.131044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:38.139528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1126/2264] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:38.143488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.153374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfd41b-7a59-4d7a-8125-fc91773cac05 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.157865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.158447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.158722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.158995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.159624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:38.159920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.161601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.172645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1139/2265] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:38.176656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.185659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04886b53-ae4e-42a9-bc03-d548be8106ce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.190266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.190854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.191130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.191402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.192077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.193939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.221674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1127/2266] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:38 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:38.225930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.236519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda1f68-17a3-436f-9e3d-723843d25a0f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.266383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:38.293567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.294166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.294445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.294722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.295341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.297278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.318744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1140/2267] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:38 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:38.323641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.334065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce62f5-2529-4583-ada2-4ec09f2506b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.338964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.339561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.339841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.340120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.340763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.342399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.352206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1128/2268] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:38.356052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.365710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15492d-ae17-4f60-9a7c-35b323a235b0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.370811 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.371405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.371684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.371962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.372602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.374214 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7103155-91d1-4ec6-89ae-5a72a2b2ec00 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.384684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1141/2269] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:38.388700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.398458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed96ac-b685-4a66-9695-dfa365aa0830 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.403036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.403666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.403946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.404221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] PATH_INFO: `/v3/projects/eb28351e15394afa975d935eea447fef/users/fa83947023284a13af9fac67cb944db2/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.419488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb28351e15394afa975d935eea447fef, user_id=fa83947023284a13af9fac67cb944db2, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.422192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.435567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1129/2270] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:38 2022] PUT /identity/v3/projects/eb28351e15394afa975d935eea447fef/users/fa83947023284a13af9fac67cb944db2/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:38.446560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f787-e87f-4198-9ed8-dc02591bdf6c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.446853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f787-e87f-4198-9ed8-dc02591bdf6c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.447128 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7f787-e87f-4198-9ed8-dc02591bdf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.474136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-94b7f787-e87f-4198-9ed8-dc02591bdf6c None None] MFA Rules not processed for user `fa83947023284a13af9fac67cb944db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:38.475322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94b7f787-e87f-4198-9ed8-dc02591bdf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:38.543891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1142/2271] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:38.555761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:38.557468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:38.575947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:38.614600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af083f58-a287-44f0-a130-63a0574c05f9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:38.618844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:38.619462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:38.619719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:38.619984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:38.628530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:38.630014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:38.645370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1130/2272] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:42.133467 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d7987-f6ff-4ad5-959f-a151fa284c96 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:42.134003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d7987-f6ff-4ad5-959f-a151fa284c96 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:42.134003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d7987-f6ff-4ad5-959f-a151fa284c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:42.157443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cb1d7987-f6ff-4ad5-959f-a151fa284c96 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:42.159045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb1d7987-f6ff-4ad5-959f-a151fa284c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:42.231144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1143/2273] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:15:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:42.279221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:42.280706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:42.362745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3625b5d-4cdd-41cd-a4fd-ba0a0ef89836 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:42.367668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:42.368332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:42.368611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:42.368879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:42.377075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:42.378672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:42.389166 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1131/2274] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:15:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:15:43.003275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1ae0-da77-42d3-b989-f760870f12db None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.003458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1ae0-da77-42d3-b989-f760870f12db None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.003897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1ae0-da77-42d3-b989-f760870f12db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.042259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-792d1ae0-da77-42d3-b989-f760870f12db None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:43.043628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-792d1ae0-da77-42d3-b989-f760870f12db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:43.052181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1144/2275] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 07:15:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:43.066850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:43.078306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882ec65-c1fe-49ec-882f-2504a1496dfc service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:43.083970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:43.084784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.085325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.085738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.095312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:43.096951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:43.106383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1132/2276] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:15:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:15:43.371304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e791ef-5e22-4b80-a39d-9e728c503b9c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.371812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e791ef-5e22-4b80-a39d-9e728c503b9c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.371901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e791ef-5e22-4b80-a39d-9e728c503b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.391112 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-28e791ef-5e22-4b80-a39d-9e728c503b9c None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:43.392566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-28e791ef-5e22-4b80-a39d-9e728c503b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:43.399749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1145/2277] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:15:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:43.411799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:43.423331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c8aa-dd76-4f32-b8c7-ccac0e67bc99 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:43.428456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:43.429189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.429501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.429840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.439884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:43.441713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:43.449873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1133/2278] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:15:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:15:43.792982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.793338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.793613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.812105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:43.813310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:43.820265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1146/2279] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 07:15:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:43.832292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:43.843149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f2469-387c-4517-b000-95f3e9376586 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:43.848153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:43.848740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:43.849019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:43.849322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:43.857753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:43.859319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94df8-1fc6-47df-89b9-b88dfd12d668 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:43.867422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1134/2280] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:15:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:15:48.091709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.093408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.134882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:48.178324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebe460-1cc8-4f69-b16e-0e1463037325 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:48.182842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:48.183478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.183755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.184028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] PATH_INFO: `/v3/users/83123e2a0aaa4f17b2f3c7dd70572658` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.185209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83123e2a0aaa4f17b2f3c7dd70572658)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:48.187110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:48.234526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83123e2a0aaa4f17b2f3c7dd70572658'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.235176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83123e2a0aaa4f17b2f3c7dd70572658'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.241954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83123e2a0aaa4f17b2f3c7dd70572658'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.242605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83123e2a0aaa4f17b2f3c7dd70572658'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.249761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83123e2a0aaa4f17b2f3c7dd70572658'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.266695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1147/2281] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:48 2022] DELETE /identity/v3/users/83123e2a0aaa4f17b2f3c7dd70572658 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:48.271206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.271669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.271669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.291062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:48.292242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.349621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1135/2282] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:48.359726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.361259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.416878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5cde8-d530-4cf5-babf-7e0ec8131ab8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:48.420489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:48.421083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.421386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.421654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.430959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:48.432374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48481b-7d03-48e0-a43d-8914c53a51f8 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:48.441604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1148/2283] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:48.609540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.611001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.628704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d4846-c9ba-47be-81eb-9ae7e77302c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:48.634130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:48.634793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.635102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.635373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] PATH_INFO: `/v3/projects/8e9f3629cb0e4cf393ddf8942c5a6946` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.636523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9f3629cb0e4cf393ddf8942c5a6946)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:48.638487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:48.640880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:48.691670 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9f3629cb0e4cf393ddf8942c5a6946', 'request_id': 'req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.698720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9f3629cb0e4cf393ddf8942c5a6946', 'request_id': 'req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.704333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9f3629cb0e4cf393ddf8942c5a6946', 'request_id': 'req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.718987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] The token cache is being invalidate because project 8e9f3629cb0e4cf393ddf8942c5a6946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:48.719279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.719894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80757b8-77f0-4b0d-bde1-eef3f301d6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:48.721421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1136/2284] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:48 2022] DELETE /identity/v3/projects/8e9f3629cb0e4cf393ddf8942c5a6946 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:48.726802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.727164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.727946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.747711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:48.748905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.796349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:48.817229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1149/2285] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:48.825587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.825897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.826174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.845512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:48.846713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:48.854640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1137/2286] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:48.858728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.868966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb967-a767-4676-853b-3f65f5723f9a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:48.873503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:48.874089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.874363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.874637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.875274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:48.875573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:48.877177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:48.889140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1150/2287] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:48.893191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.902290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0793a1b-fddc-4402-96d8-d687fb561bbe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:48.906969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:48.907608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:48.907884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:48.908158 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:48.908791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:48.910725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:48.937462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1138/2288] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:48 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:48.941482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:48.951862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacd0b2-b962-4726-a5a0-212bb7767920 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:49.005138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:49.005741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.006015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.006284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.006892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:49.009695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:49.029647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1151/2289] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:48 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:49.033768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:49.043882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01bce0-da9f-4b16-aa54-e78418d87a6e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:49.048443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:49.049034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.049347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.049622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.050258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:49.051857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:49.062304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1139/2290] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:49.066323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:49.076637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024b51a-4091-4fd6-ae66-b4a83ff33840 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:49.081279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:49.081896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.082212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.082491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.083129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:49.084715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84b360-9c3c-434e-81f6-acb1cde700cd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:49.099968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1152/2291] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:49.101208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:49.112284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a12f31-3fcb-4e0b-b331-e95ed9eb1d29 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:49.118075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:49.118747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.119027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.119300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] PATH_INFO: `/v3/projects/670ca9b5239a4c5f9aa30faca7a4e519/users/8a537ff040dc4559b103696638cdd9d5/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.134878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670ca9b5239a4c5f9aa30faca7a4e519, user_id=8a537ff040dc4559b103696638cdd9d5, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:49.137641 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:49.151647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1140/2292] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:49 2022] PUT /identity/v3/projects/670ca9b5239a4c5f9aa30faca7a4e519/users/8a537ff040dc4559b103696638cdd9d5/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:49.156814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.157105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.157409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.176586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:49.177801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:49.236861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1153/2293] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:49.247160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:49.248874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:49.309998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e762a8-f5a9-43b5-bb01-a559132b6dd7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:49.314211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:49.314891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:49.315178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:49.315430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:49.325173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:49.326601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976052d-e584-4a26-9d3b-f3ecc819a0e4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:49.336421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1141/2294] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:52.123056 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:52.125153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:52.138314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1485601-3d25-41a8-8b7f-facb832e0663 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:52.144984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:52.145883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:52.146346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:52.146805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] PATH_INFO: `/v3/users/061052054914431db212078ee32853cc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:52.148429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=061052054914431db212078ee32853cc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:52.151351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:52.241219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '061052054914431db212078ee32853cc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:52.242472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '061052054914431db212078ee32853cc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:52.245406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:52.253264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '061052054914431db212078ee32853cc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:52.254264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '061052054914431db212078ee32853cc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:52.272900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '061052054914431db212078ee32853cc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:52.300819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1154/2295] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:52 2022] DELETE /identity/v3/users/061052054914431db212078ee32853cc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:52.302108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:52.302601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:52.302714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:52.355126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:52.357289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:52.401197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:52.463151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1142/2296] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:52.487076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:52.488544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:52.635272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925856c9-820b-41ea-afc2-e729b38257ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:52.648452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:52.649818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:52.651628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:52.652057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:52.673047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:52.674734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:52.688422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1155/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:53.156730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:53.158440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:53.177065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48788d63-c6db-4132-9bdb-033d90174060 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:53.186610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:53.187280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:53.187560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:53.187838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] PATH_INFO: `/v3/projects/912fa846b51647678e981950e8741d9a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:53.189310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] RBAC: Authorizing `identity:delete_project(project_id=912fa846b51647678e981950e8741d9a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:53.191333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:53.263310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '912fa846b51647678e981950e8741d9a', 'request_id': 'req-83287dfc-3599-4296-b4f4-51be2f39997e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.272076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '912fa846b51647678e981950e8741d9a', 'request_id': 'req-83287dfc-3599-4296-b4f4-51be2f39997e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.278585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '912fa846b51647678e981950e8741d9a', 'request_id': 'req-83287dfc-3599-4296-b4f4-51be2f39997e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.293408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] The token cache is being invalidate because project 912fa846b51647678e981950e8741d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:53.293671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.295070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.297220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1143/2298] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:53 2022] DELETE /identity/v3/projects/912fa846b51647678e981950e8741d9a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:53.301077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:53.312379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b1d02-2d72-4bbd-a1cb-279de38cf389 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:53.390140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:53.390945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:53.391479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:53.392030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] PATH_INFO: `/v3/users/04e25b8b85bc46cd99890fc1869a3962` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:53.393415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e25b8b85bc46cd99890fc1869a3962)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:53.395486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:53.474797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e25b8b85bc46cd99890fc1869a3962'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.475440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e25b8b85bc46cd99890fc1869a3962'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.483950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e25b8b85bc46cd99890fc1869a3962'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.484696 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e25b8b85bc46cd99890fc1869a3962'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.494145 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a67e9eff-f703-4d29-9ee0-169c3388f6e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e25b8b85bc46cd99890fc1869a3962'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:53.504191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:53.520124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1156/2299] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:53 2022] DELETE /identity/v3/users/04e25b8b85bc46cd99890fc1869a3962 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:53.946204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:53.947774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:53.966634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287dfc-3599-4296-b4f4-51be2f39997e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.036362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.037006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.037314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.037591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] PATH_INFO: `/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.038743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ffaebc3a1f499e91a32b15ff45ea74)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.041272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.115547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ffaebc3a1f499e91a32b15ff45ea74', 'request_id': 'req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:54.150620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ffaebc3a1f499e91a32b15ff45ea74', 'request_id': 'req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:54.155110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:54.169595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ffaebc3a1f499e91a32b15ff45ea74', 'request_id': 'req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:54.196613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] The token cache is being invalidate because project e0ffaebc3a1f499e91a32b15ff45ea74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:54.196924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:54.198064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dbdbfd9f-8e3b-4286-9074-3e3c3b765218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:54.200506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1144/2300] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:53 2022] DELETE /identity/v3/projects/e0ffaebc3a1f499e91a32b15ff45ea74 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:54.205442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.205769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.206052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.232272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:54.235612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:54.310449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1157/2301] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:54.319743 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.320231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.320711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.353375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:54.355965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:54.365718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1145/2302] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:54.369535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.382481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1287244-938e-4d05-9ac4-835a00a0244c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.390860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.392801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.393098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.393412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.394114 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:15:54.394710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.396398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.413728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1158/2303] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:15:54.418252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.431251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82fb61-a4ef-4de3-b013-d3cc65f2d4df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.451749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.454488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.454665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.455468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.457294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.459383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.501164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1146/2304] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/projects => generated 365 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:54.504706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.517906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c16dd3-d601-4457-b52d-683a04118ce7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.588732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.589714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.590191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.590655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.591475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.593823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.622631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1159/2305] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:54.626587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.638868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c9ce-1ec8-4530-8ee0-13bab40ae281 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.643673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.644362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.644644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.644913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.645640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.647199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.658646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1147/2306] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:54.665333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.677952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa5344-8f02-4b1e-ba74-9141fb90b632 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.690300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.691172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.691635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.692981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.693893 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.696388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082559-54b9-4606-802d-8bc0d89832c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.721616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1160/2307] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:54.725791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:54.740101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceb99d-286c-4e49-9a66-b1aabf089ddd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:54.749789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:54.750556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.750907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.751287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] PATH_INFO: `/v3/projects/fd0f9d90a42d47cf858ba142429622ad/users/7832a80214cb4680a26b92378b4ce35c/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.773201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0f9d90a42d47cf858ba142429622ad, user_id=7832a80214cb4680a26b92378b4ce35c, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:54.776236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:54.790022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1148/2308] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:54 2022] PUT /identity/v3/projects/fd0f9d90a42d47cf858ba142429622ad/users/7832a80214cb4680a26b92378b4ce35c/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:54.801871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:54.802163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:54.802443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:54.835245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] MFA Rules not processed for user `7832a80214cb4680a26b92378b4ce35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:54.836776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:54.891563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:54.937586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1161/2309] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:54.939076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.097342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffcd09-edf6-4691-8c4d-892dee74fce7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.104396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.105378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.105807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.106279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.107079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.109783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abbdcc-864f-40d8-8d37-469342a6afc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.147483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1149/2310] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:54 2022] POST /identity/v3/projects => generated 369 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:55.151306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.162832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36fe7d5-bbaf-41ac-a723-4abf9a8bdf94 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.206521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.207311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.207748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.245723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.246454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.246842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.247396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.248154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.250454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.253612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] MFA Rules not processed for user `8a537ff040dc4559b103696638cdd9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:55.254988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:55.289991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1162/2311] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:55 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:55.294328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.312536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940148-7875-4c02-b644-5e720db90660 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.318285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.319125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.319559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.320074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.322009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.324494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.336289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1163/2312] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:55.341324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.353447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df1fbe-fae1-494f-9eb2-10d0bc2bb078 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.359224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.360301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.360830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.361474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] PATH_INFO: `/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec/users/20e42567ab2347b2ba8a1bdf634bd7f6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.371155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:55.380052 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b564fd6b2e448cfb4f76640c708e2ec, user_id=20e42567ab2347b2ba8a1bdf634bd7f6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.383642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.385971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1150/2313] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:15:55.391624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.394085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:55.404928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1164/2314] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:55 2022] PUT /identity/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec/users/20e42567ab2347b2ba8a1bdf634bd7f6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:15:55.408801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.411804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b533f92-2b0e-46df-b8b6-7ae4756faa6c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.481416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214eae2d-71e9-4154-97a9-b38144a5e2bb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.483983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.484596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.484877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.485353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.485912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.488142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.488495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.488901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.489291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.489630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.490412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.493154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.506061 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1165/2315] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:15:55.511068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.523657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33f287-91bd-47a7-8283-1b2cc1cc26ac admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.530018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.530872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.531269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.535003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1151/2316] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:15:55 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:15:55.535161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] PATH_INFO: `/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec/users/20e42567ab2347b2ba8a1bdf634bd7f6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.535161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b564fd6b2e448cfb4f76640c708e2ec, user_id=20e42567ab2347b2ba8a1bdf634bd7f6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.535787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.536922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85033a21-da4b-41b1-b040-d23f770086b8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.551478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1166/2317] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:55 2022] PUT /identity/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec/users/20e42567ab2347b2ba8a1bdf634bd7f6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:55.565798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.566243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.567258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.606189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] MFA Rules not processed for user `20e42567ab2347b2ba8a1bdf634bd7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:55.607427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:55.619116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:55.623004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7822d-4e3a-4e08-ba2c-6a6689d6f8ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.697243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.697864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.698136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.698405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.699044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.701086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.707922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1167/2318] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:55.719051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.720459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:55.762242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1152/2319] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:15:55 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:15:55.766471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.788723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa7c8-873c-4b87-bea1-ccf54d393cb0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.803888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.804518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.805332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.806119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.806277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.810622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.828479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1153/2320] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:55.828702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220daf2c-662e-43a1-9b68-0ddb1d6ff4ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.829535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.840592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.840839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea17c-30c5-4e2a-af51-a88df235b1d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:55.844910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.845062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.845492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.853622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:55.858211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:55.860273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:55.860706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] PATH_INFO: `/v3/projects/510133b0ebba4c1684bf0f56ca75cc4a/users/b7616922f213460d970aee532a7a09c1/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:55.869252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.871441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.874310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:55.887851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1168/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:55.901973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510133b0ebba4c1684bf0f56ca75cc4a, user_id=b7616922f213460d970aee532a7a09c1, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:55.904711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:55.923021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1154/2322] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:55 2022] PUT /identity/v3/projects/510133b0ebba4c1684bf0f56ca75cc4a/users/b7616922f213460d970aee532a7a09c1/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:55.928519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:55.943268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8f08-7f4a-45e0-949a-2c96fb30061a service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:56.035489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:56.036153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:56.036428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:56.036688 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:56.037628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:56.039998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:56.056453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1169/2323] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:15:56.062051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:56.075534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a67e-28ab-4ca1-80be-028e33d06709 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:56.080375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:56.081036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:56.081367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:56.081650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] PATH_INFO: `/v3/projects/510133b0ebba4c1684bf0f56ca75cc4a/users/b7616922f213460d970aee532a7a09c1/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:56.083701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510133b0ebba4c1684bf0f56ca75cc4a, user_id=b7616922f213460d970aee532a7a09c1, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:56.086656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67af6b-2d98-43d6-9aff-a5d19eae9774 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:56.103885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1155/2324] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:15:56 2022] PUT /identity/v3/projects/510133b0ebba4c1684bf0f56ca75cc4a/users/b7616922f213460d970aee532a7a09c1/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:58.235024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:58.248137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b08f654-8c75-40cc-99ff-62d1a7bb39a9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:58.325206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:58.345317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:58.346094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:58.346878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:58.347409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] PATH_INFO: `/v3/users/7832a80214cb4680a26b92378b4ce35c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:58.352479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7832a80214cb4680a26b92378b4ce35c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:58.356035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:58.487445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7832a80214cb4680a26b92378b4ce35c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:58.488396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7832a80214cb4680a26b92378b4ce35c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:58.504389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7832a80214cb4680a26b92378b4ce35c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:58.511246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7832a80214cb4680a26b92378b4ce35c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:58.521484 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7832a80214cb4680a26b92378b4ce35c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:58.560277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1170/2325] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:58 2022] DELETE /identity/v3/users/7832a80214cb4680a26b92378b4ce35c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:58.565194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:58.566242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:58.567025 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:58.599760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:15:58.601432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:58.745162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1156/2326] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:15:58.759769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:58.761647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:58.887850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cc4ad-b18c-4acb-9a71-876d4be2dcc2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:58.892328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:58.893169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:58.894493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:58.894493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:58.926785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:58.936977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:58.948476 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1171/2327] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:15:59.322893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:59.328940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:15:59.376982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bb230-5c9c-4e5a-a4f2-c14fc0c79c34 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:59.383730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:59.384629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:59.385203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:59.385688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] PATH_INFO: `/v3/projects/fd0f9d90a42d47cf858ba142429622ad` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:59.387123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0f9d90a42d47cf858ba142429622ad)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:59.389866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:59.591509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0f9d90a42d47cf858ba142429622ad', 'request_id': 'req-36b49270-e5c3-4ea3-af86-676f10bc8048'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.603359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0f9d90a42d47cf858ba142429622ad', 'request_id': 'req-36b49270-e5c3-4ea3-af86-676f10bc8048'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.614450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0f9d90a42d47cf858ba142429622ad', 'request_id': 'req-36b49270-e5c3-4ea3-af86-676f10bc8048'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.653437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] The token cache is being invalidate because project fd0f9d90a42d47cf858ba142429622ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:15:59.654793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.658560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.662544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1157/2328] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:15:59 2022] DELETE /identity/v3/projects/fd0f9d90a42d47cf858ba142429622ad => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:15:59.666474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:15:59.679640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc30ac-7bfb-4cb4-9878-3f81e464d671 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:15:59.764443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:15:59.765397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:15:59.765578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:15:59.765848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] PATH_INFO: `/v3/users/20e42567ab2347b2ba8a1bdf634bd7f6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:15:59.767401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20e42567ab2347b2ba8a1bdf634bd7f6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:15:59.769762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:15:59.794509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:15:59.867106 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20e42567ab2347b2ba8a1bdf634bd7f6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.868073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20e42567ab2347b2ba8a1bdf634bd7f6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.885927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20e42567ab2347b2ba8a1bdf634bd7f6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.886716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20e42567ab2347b2ba8a1bdf634bd7f6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.906900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd2d3253-8050-4b0e-8593-ed14ac115c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20e42567ab2347b2ba8a1bdf634bd7f6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:15:59.944631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1172/2329] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:15:59 2022] DELETE /identity/v3/users/20e42567ab2347b2ba8a1bdf634bd7f6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:00.383629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:00.385565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:00.410253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49270-e5c3-4ea3-af86-676f10bc8048 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:00.492211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:00.514911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:00.515873 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:00.516473 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:00.516991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] PATH_INFO: `/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:00.532965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b564fd6b2e448cfb4f76640c708e2ec)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:00.537888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:00.698378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b564fd6b2e448cfb4f76640c708e2ec', 'request_id': 'req-576138ac-ba90-461b-91b8-67352344df41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:00.732321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b564fd6b2e448cfb4f76640c708e2ec', 'request_id': 'req-576138ac-ba90-461b-91b8-67352344df41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:00.754389 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b564fd6b2e448cfb4f76640c708e2ec', 'request_id': 'req-576138ac-ba90-461b-91b8-67352344df41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:00.805506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] The token cache is being invalidate because project 0b564fd6b2e448cfb4f76640c708e2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:00.805810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:00.806487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-576138ac-ba90-461b-91b8-67352344df41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:00.808550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1158/2330] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:00 2022] DELETE /identity/v3/projects/0b564fd6b2e448cfb4f76640c708e2ec => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:00.814983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:00.815633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:00.827736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:00.865340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:00.869219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:00.986024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1173/2331] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:00.993019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:00.993341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:00.993612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.016807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:01.018450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:01.027781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1159/2332] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:01.032976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.044688 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253f76b-ca55-4b9f-aef0-40c9a9e9d54a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.049547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.050777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.050777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.051065 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.051570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:01.051911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.053626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.072791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1174/2333] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:16:01.077161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.096805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df94f15-791d-4389-b309-89f2b87d9f01 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.104578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.106081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.106993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.107643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.110176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.114309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.159444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1160/2334] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:01.193313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.217689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4a0d2-4923-4049-a0bf-7b0563662e94 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.329534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.329534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.329534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.329534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.329887 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.330262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.365426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1175/2335] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:01 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:01.366672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.382432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c75570-256d-4fd8-b099-51a203a85d73 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.396845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.397505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.397780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.398048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.398776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.400727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.422522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1161/2336] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:01.426959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.441867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:01.451689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab184be-98bf-4291-96bb-e316fcf8c496 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.467964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.469230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.473014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.473530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.476578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.480370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d003c-6517-43ac-b499-64a844a830cf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.505192 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1176/2337] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:01.510005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.524558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d9c69-84c0-434c-a503-09af4d51753e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.530484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.531156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.531432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.531780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] PATH_INFO: `/v3/projects/a12b17dc50b54786ab086adff77d3cea/users/27f4f85af2b64f6196e8064c5cfb7a3b/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.568615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12b17dc50b54786ab086adff77d3cea, user_id=27f4f85af2b64f6196e8064c5cfb7a3b, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.571463 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.616206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1162/2338] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:01 2022] PUT /identity/v3/projects/a12b17dc50b54786ab086adff77d3cea/users/27f4f85af2b64f6196e8064c5cfb7a3b/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:01.617883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.619272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.619272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.664984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] MFA Rules not processed for user `27f4f85af2b64f6196e8064c5cfb7a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:01.666336 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:01.748684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1177/2339] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:01.749725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.756496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:01.759938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:01.766978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:01.853694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561adee8-5ced-4537-857e-854c50ceeba7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.860847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.861643 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.862040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.862383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.873418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.875036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3e054-5be7-4d80-bb71-b2f707673db2 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.890718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1178/2340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:01.900519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc03f9-9ea6-4f8f-9634-df37b2abd4ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:01.907595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:01.908903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:01.909378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:01.909795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] PATH_INFO: `/v3/users/41df4be7d90e4f2b8f2147489ea6e2e7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:01.911699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41df4be7d90e4f2b8f2147489ea6e2e7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:01.914433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:01.947700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:02.012361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41df4be7d90e4f2b8f2147489ea6e2e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:02.034454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41df4be7d90e4f2b8f2147489ea6e2e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:02.064270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41df4be7d90e4f2b8f2147489ea6e2e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:02.065367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41df4be7d90e4f2b8f2147489ea6e2e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:02.097684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41df4be7d90e4f2b8f2147489ea6e2e7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:02.123100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1163/2341] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:01 2022] DELETE /identity/v3/users/41df4be7d90e4f2b8f2147489ea6e2e7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:02.127639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791558a-9d91-46d1-b3d1-99cac1162f46 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:02.128416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791558a-9d91-46d1-b3d1-99cac1162f46 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:02.130460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791558a-9d91-46d1-b3d1-99cac1162f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:02.205999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d791558a-9d91-46d1-b3d1-99cac1162f46 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:02.208989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d791558a-9d91-46d1-b3d1-99cac1162f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:02.452213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1179/2342] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:02.462872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:02.464512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:02.608433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356b7d4-07b3-469a-a702-603c744c52e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:02.616601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:02.617495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:02.618936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:02.621521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:02.656353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:02.658238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:02.672104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1164/2343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:02.880038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:02.881221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:02.881499 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:02.931104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] MFA Rules not processed for user `b7616922f213460d970aee532a7a09c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:02.932473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:02.948871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:03.045512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1180/2344] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:03.056307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:03.057875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:03.074804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:03.077419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:03.106691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b52b0-22e1-4f13-ac19-1a62a318e432 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:03.113283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:03.114095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:03.115530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:03.116111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] PATH_INFO: `/v3/projects/f71bcba137f947e28bd7b8c0c8388ea7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:03.126071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71bcba137f947e28bd7b8c0c8388ea7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:03.128301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:03.160191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56850-ffb6-4897-b3da-a0a6b039db2f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:03.165396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c262262-f659-45a8-913d-571103198e94 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:03.166306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c262262-f659-45a8-913d-571103198e94 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:03.168300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c262262-f659-45a8-913d-571103198e94 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:03.168968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c262262-f659-45a8-913d-571103198e94 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:03.172032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c262262-f659-45a8-913d-571103198e94 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:03.207073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71bcba137f947e28bd7b8c0c8388ea7', 'request_id': 'req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:03.221604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71bcba137f947e28bd7b8c0c8388ea7', 'request_id': 'req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:03.228945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71bcba137f947e28bd7b8c0c8388ea7', 'request_id': 'req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:03.240289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] The token cache is being invalidate because project f71bcba137f947e28bd7b8c0c8388ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:03.240593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:03.241704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58bd21d5-ba17-40e7-b9e3-966f5d6ca0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:03.246294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1181/2345] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:03 2022] DELETE /identity/v3/projects/f71bcba137f947e28bd7b8c0c8388ea7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:03.257615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:03.257615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c262262-f659-45a8-913d-571103198e94 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:03.257615 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c262262-f659-45a8-913d-571103198e94 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:03.269598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1165/2346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:03.543477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:03.543946 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:03.544421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:03.607126 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:03.612852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:03.800158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1182/2347] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:03.815690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c262262-f659-45a8-913d-571103198e94 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:03.817386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c262262-f659-45a8-913d-571103198e94 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:03.940787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c262262-f659-45a8-913d-571103198e94 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:04.016031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:04.016732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:04.017006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:04.017347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:04.034029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:04.035717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd7afd-9728-4c21-a660-4f7ae4092746 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:04.049828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1166/2348] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:16:04.869060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:04.871094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:04.959717 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:05.010800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6cb39-b9ae-4da6-a6eb-94ba3ee3ab37 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:05.025608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:05.026277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:05.026560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:05.026838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] PATH_INFO: `/v3/users/22b9e583ba7d4e1ca531ec12f5c7c199` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:05.029909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b9e583ba7d4e1ca531ec12f5c7c199)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:05.035294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:05.169694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b9e583ba7d4e1ca531ec12f5c7c199'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.170403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b9e583ba7d4e1ca531ec12f5c7c199'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.182636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b9e583ba7d4e1ca531ec12f5c7c199'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.184824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b9e583ba7d4e1ca531ec12f5c7c199'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.201453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b9e583ba7d4e1ca531ec12f5c7c199'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.230844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1183/2349] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:04 2022] DELETE /identity/v3/users/22b9e583ba7d4e1ca531ec12f5c7c199 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:05.235296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:05.236210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:05.237157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:05.260717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:05.262530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:05.361965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1167/2350] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:05.372500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:05.374023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:05.439944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0bb9b-db36-46fc-89ae-80b2b9f9d227 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:05.443617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:05.444216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:05.444490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:05.444760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:05.457477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:05.459057 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036c2f0-5576-454b-9c88-2e441b5e6661 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:05.475730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1184/2351] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:16:05.844642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:05.847064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:05.867536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72932ec1-e5d0-465e-aa73-c3b451acea03 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:05.872630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:05.873249 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:05.873521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:05.873796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] PATH_INFO: `/v3/projects/e75c8121aefb4a0081cba2e575b8617b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:05.874930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75c8121aefb4a0081cba2e575b8617b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:05.877147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:05.908682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:05.941462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75c8121aefb4a0081cba2e575b8617b', 'request_id': 'req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.951531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75c8121aefb4a0081cba2e575b8617b', 'request_id': 'req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.958502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75c8121aefb4a0081cba2e575b8617b', 'request_id': 'req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.979050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] The token cache is being invalidate because project e75c8121aefb4a0081cba2e575b8617b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:05.979454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.980049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c39b78b1-3fcc-4f1b-9b5f-25389d1ab896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:05.982518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1168/2352] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:05 2022] DELETE /identity/v3/projects/e75c8121aefb4a0081cba2e575b8617b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:05.987962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:05.988545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:05.988545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.043154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:06.044638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:06.118362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1185/2353] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:06.125932 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.126316 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.126664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.163677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:06.165498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:06.179618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1169/2354] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:06.183269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.188703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:06.195092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded83465-a0ae-4537-85b1-93f1dcb1cfa4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.208607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.209283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.209564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.209864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.210528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:06.210833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.213920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.236321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1186/2355] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:16:06.237695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.249359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4b508-ed9e-490a-9d1f-e5b0df7b3ebd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.254694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.255558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.256041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.256514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.257639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.262683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.329886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1170/2356] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:06 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:06.330833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.349097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349126f9-4909-4398-bed4-1e0c1e7ce763 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.488966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.489615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.489891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.490160 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.490793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.492861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.544202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1187/2357] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:06 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:16:06.545776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.558655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845071-8751-4dc4-8315-40fa4e1b9334 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.568424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.569508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.570260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.571082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.571739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.573412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.596864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1171/2358] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:06.600731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.619755 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f1899-8be4-4cbb-8404-b28dc67cbd12 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.637483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.638273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.641777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.642291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.643095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.645399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593cc8a-6771-4158-aa2c-c7b291ba17b9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.668765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1188/2359] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:06.673571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.685195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ff4f3-3f09-4057-aa35-17d45c9e41d3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:06.692263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:06.693046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.693434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.693785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] PATH_INFO: `/v3/projects/5959f07025da467d8de0ac85b80be04c/users/e61c3bf2af8745dca92d43d3f78bda06/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.715895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5959f07025da467d8de0ac85b80be04c, user_id=e61c3bf2af8745dca92d43d3f78bda06, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:06.718930 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:06.741089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1172/2360] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:06 2022] PUT /identity/v3/projects/5959f07025da467d8de0ac85b80be04c/users/e61c3bf2af8745dca92d43d3f78bda06/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:06.762760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad23f-c5f3-4202-8cf0-7c31e8f9d69d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:06.763323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad23f-c5f3-4202-8cf0-7c31e8f9d69d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:06.763788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fad23f-c5f3-4202-8cf0-7c31e8f9d69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:06.804228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-33fad23f-c5f3-4202-8cf0-7c31e8f9d69d None None] MFA Rules not processed for user `e61c3bf2af8745dca92d43d3f78bda06`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:06.805690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-33fad23f-c5f3-4202-8cf0-7c31e8f9d69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:06.896098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1189/2361] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:06.909653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:06.912086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:07.054327 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61034f7b-3852-4798-ac02-b824d43feb25 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:07.058697 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:07.059521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:07.059806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:07.060081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:07.069487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:07.071644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:07.083321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1173/2362] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:09.861771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d79647-8d26-49aa-92aa-76f335f3abce None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:09.863248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d79647-8d26-49aa-92aa-76f335f3abce None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:09.863750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d79647-8d26-49aa-92aa-76f335f3abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:09.890419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e1d79647-8d26-49aa-92aa-76f335f3abce None None] MFA Rules not processed for user `1cc519da0b5d40268b29bb1bef89049b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:09.895526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e1d79647-8d26-49aa-92aa-76f335f3abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:10.038580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1190/2363] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:11.061483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:11.062034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:11.133861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:11.161604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d43d8-e9e8-470a-a085-0850c5992761 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:11.167659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:11.168998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:11.169382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:11.170982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] PATH_INFO: `/v3/users/27f4f85af2b64f6196e8064c5cfb7a3b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:11.172202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f4f85af2b64f6196e8064c5cfb7a3b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:11.174206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:11.265971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f4f85af2b64f6196e8064c5cfb7a3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:11.276886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f4f85af2b64f6196e8064c5cfb7a3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:11.288460 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f4f85af2b64f6196e8064c5cfb7a3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:11.289359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f4f85af2b64f6196e8064c5cfb7a3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:11.359350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f4f85af2b64f6196e8064c5cfb7a3b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:11.369315 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1174/2364] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:11 2022] DELETE /identity/v3/users/27f4f85af2b64f6196e8064c5cfb7a3b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:11.393998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:11.399780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:11.400307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:11.408631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:11.435835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:11.438631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:11.517293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1191/2365] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:11.530915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:11.536065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:11.614310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464bfa7-a91a-4b6b-8587-c3fc11ab928e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:11.623207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:11.623839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:11.624119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:11.624390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:11.638523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:11.640104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf450ef-2911-43f8-8451-2344d9b9e355 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:11.651584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1175/2366] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:12.014723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.016988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:12.058754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794436d-44d7-4ff7-8586-91ad10893888 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.064124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.064770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.065042 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.065351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] PATH_INFO: `/v3/projects/a12b17dc50b54786ab086adff77d3cea` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.066712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12b17dc50b54786ab086adff77d3cea)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.068656 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.128546 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12b17dc50b54786ab086adff77d3cea', 'request_id': 'req-184516ac-f09e-4be2-8de9-8ffc63d0c17d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:12.142118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12b17dc50b54786ab086adff77d3cea', 'request_id': 'req-184516ac-f09e-4be2-8de9-8ffc63d0c17d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:12.152762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12b17dc50b54786ab086adff77d3cea', 'request_id': 'req-184516ac-f09e-4be2-8de9-8ffc63d0c17d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:12.169206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] The token cache is being invalidate because project a12b17dc50b54786ab086adff77d3cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:12.169593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:12.170416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-184516ac-f09e-4be2-8de9-8ffc63d0c17d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:12.173775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1192/2367] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:12 2022] DELETE /identity/v3/projects/a12b17dc50b54786ab086adff77d3cea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:12.176801 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.177357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.177459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.205159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:12.206409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:12.331536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1176/2368] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:12.339663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.340392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.340681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.372875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:12.374226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:12.384557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1193/2369] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:16:12.389391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.403269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c742-57df-43db-8380-b0127d6c1854 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.408947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.409968 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.410612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.411415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.414909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:12.416958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.419946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.431855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:12.441920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1177/2370] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:16:12.442358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.462168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5027f-b6f3-4cac-9c71-145db05c168d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.467124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.467742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.468170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.468450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.469103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.473658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.513073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1194/2371] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:12 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:16:12.517016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.530769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309944cb-ae29-46cc-beba-270cb5dce3ed admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.608992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.610094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.610671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.614323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.615973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.619397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.654475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1178/2372] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:12 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:12.662408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.678283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067befa-4e78-4646-a710-6437abd621c6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.684855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.689200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.689890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.690320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.692140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.695991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.708172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1195/2373] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:12.715982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.741307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca33f0e-6d88-49e0-b5ba-590e4c1e5d0f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.746683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.749566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.750082 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.756786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.756786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.756786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1438-74e2-4c12-944a-25bf3c0dc4b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.779886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:12.781069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1179/2374] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:12.806262 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da655217-f4ed-4816-a398-571a9bef9253 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:12.813788 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:12.814740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.815241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.815637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] PATH_INFO: `/v3/projects/bbd9494c68544101b8cd85dfc1c57d3a/users/5eb641e1a3954b0abf5d33a1c83fd4bc/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.841037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd9494c68544101b8cd85dfc1c57d3a, user_id=5eb641e1a3954b0abf5d33a1c83fd4bc, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:12.844495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:12.871999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1196/2375] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:12 2022] PUT /identity/v3/projects/bbd9494c68544101b8cd85dfc1c57d3a/users/5eb641e1a3954b0abf5d33a1c83fd4bc/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:12.885759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:12.887289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:12.887579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:12.929153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] MFA Rules not processed for user `5eb641e1a3954b0abf5d33a1c83fd4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:12.930653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:13.037841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1180/2376] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:13.038320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.078748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:13.126992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fe3ef-1cb8-4df6-96c6-54b641b35351 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.133331 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.134068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.134346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.134619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.135252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.137215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.176595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1197/2377] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:13 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:13.180794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.193780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141fc2a-07e8-43a2-8182-44aa51ec9a9e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.270879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.271508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.271787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.272060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.272827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.277501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.300346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:13.309184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1181/2378] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:13 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:13.312620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.328449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbaeed-77cf-49f6-8ac8-c519a372f875 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.336869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.337702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.338081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.338459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.339205 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.341016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.358937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1198/2379] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:13.363111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.376756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dabfa-f15e-47bb-bcd7-2453d67f5fb5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.385912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.386897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.387360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.387821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] PATH_INFO: `/v3/projects/4a738534697947c8a4c15a4eb32d2072/users/6420241d029b4843a0008853eb93579a/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.414558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a738534697947c8a4c15a4eb32d2072, user_id=6420241d029b4843a0008853eb93579a, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.417859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.437833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1182/2380] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:13 2022] PUT /identity/v3/projects/4a738534697947c8a4c15a4eb32d2072/users/6420241d029b4843a0008853eb93579a/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:13.441639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.452503 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ef5ec-7c25-42cf-82d9-8c48cd8a0aa3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.549830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.552508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.552868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.553226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.553944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.555818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0142-2459-44aa-b82f-905186dc7806 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.573164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1199/2381] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:13.574894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.588386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733029d0-b2f3-4f4a-b523-0d5d93dcd108 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.596363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.597673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.598028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.598519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] PATH_INFO: `/v3/projects/4a738534697947c8a4c15a4eb32d2072/users/6420241d029b4843a0008853eb93579a/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.600961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a738534697947c8a4c15a4eb32d2072, user_id=6420241d029b4843a0008853eb93579a, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:13.604894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:13.616504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1183/2382] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:13 2022] PUT /identity/v3/projects/4a738534697947c8a4c15a4eb32d2072/users/6420241d029b4843a0008853eb93579a/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:13.626666 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859ba-30bc-448b-99ca-20780ccff24f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.627228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859ba-30bc-448b-99ca-20780ccff24f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.627668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859ba-30bc-448b-99ca-20780ccff24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:13.668209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eab859ba-30bc-448b-99ca-20780ccff24f None None] MFA Rules not processed for user `6420241d029b4843a0008853eb93579a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:13.669896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eab859ba-30bc-448b-99ca-20780ccff24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:13.792510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1200/2383] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:13.809535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:13.812536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:13.916493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc371b52-2aad-4773-9b90-5c6463a612ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:13.926104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:13.926730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:13.927003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:13.927270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:14.029929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:14.031433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:14.056109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1184/2384] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:16.082972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b933d5-05c6-42af-b0bd-5da28efedd91 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:16.083445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b933d5-05c6-42af-b0bd-5da28efedd91 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:16.083629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b933d5-05c6-42af-b0bd-5da28efedd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:16.109771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a8b933d5-05c6-42af-b0bd-5da28efedd91 None None] MFA Rules not processed for user `8a537ff040dc4559b103696638cdd9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:16.112280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a8b933d5-05c6-42af-b0bd-5da28efedd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:16.130527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:16.202071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1201/2385] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:16.906343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:16.981580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2afa1-dc96-4fcb-803d-0806d552526a service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:16.986591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:16.987237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:16.988095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:16.988095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] PATH_INFO: `/v3/users/5eb641e1a3954b0abf5d33a1c83fd4bc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:16.989075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb641e1a3954b0abf5d33a1c83fd4bc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:16.991064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:17.019925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:17.070603 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb641e1a3954b0abf5d33a1c83fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.078984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb641e1a3954b0abf5d33a1c83fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.090719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb641e1a3954b0abf5d33a1c83fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.091532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb641e1a3954b0abf5d33a1c83fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.107402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb641e1a3954b0abf5d33a1c83fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.118935 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1185/2386] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:16 2022] DELETE /identity/v3/users/5eb641e1a3954b0abf5d33a1c83fd4bc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:17.120931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:17.121431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:17.121626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:17.145669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:17.146990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:17.230857 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1202/2387] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:17.240383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:17.242739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:17.347560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e11ad-75fa-4c72-a95c-fe6111223637 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:17.351210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:17.352480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:17.353584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:17.353584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:17.363194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:17.365168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:17.375574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1186/2388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:16:17.766118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:17.773215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:17.803412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a766-c4d2-4b57-8915-f8c25dac216a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:17.818799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:17.819592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:17.819876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:17.820169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] PATH_INFO: `/v3/projects/bbd9494c68544101b8cd85dfc1c57d3a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:17.821593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd9494c68544101b8cd85dfc1c57d3a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:17.823603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:17.904136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd9494c68544101b8cd85dfc1c57d3a', 'request_id': 'req-711a747c-2cd5-4242-bfc0-5945e94fa19a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.915028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd9494c68544101b8cd85dfc1c57d3a', 'request_id': 'req-711a747c-2cd5-4242-bfc0-5945e94fa19a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.924320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd9494c68544101b8cd85dfc1c57d3a', 'request_id': 'req-711a747c-2cd5-4242-bfc0-5945e94fa19a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.942001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] The token cache is being invalidate because project bbd9494c68544101b8cd85dfc1c57d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:17.951736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.952584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:17.958781 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1203/2389] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:17 2022] DELETE /identity/v3/projects/bbd9494c68544101b8cd85dfc1c57d3a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:17.963184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:17.979551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aa111-92b3-4cd4-8415-e03fa1d1a1f5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:18.090241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:18.090241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:18.090241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:18.090241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] PATH_INFO: `/v3/users/6420241d029b4843a0008853eb93579a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:18.091072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6420241d029b4843a0008853eb93579a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:18.093163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:18.176490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:18.192878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6420241d029b4843a0008853eb93579a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:18.201710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6420241d029b4843a0008853eb93579a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:18.210611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6420241d029b4843a0008853eb93579a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:18.211421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6420241d029b4843a0008853eb93579a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:18.235676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b9b170b3-c059-4a16-b14e-ab1be5b68574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6420241d029b4843a0008853eb93579a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:18.243844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1187/2390] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:17 2022] DELETE /identity/v3/users/6420241d029b4843a0008853eb93579a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:19.068389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.070515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:19.098659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a747c-2cd5-4242-bfc0-5945e94fa19a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.175347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:19.203551 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.204628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.205346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.206279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] PATH_INFO: `/v3/projects/4a738534697947c8a4c15a4eb32d2072` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.208687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a738534697947c8a4c15a4eb32d2072)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.212531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.276779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a738534697947c8a4c15a4eb32d2072', 'request_id': 'req-bc42e217-60dd-4ed8-814d-f22300615bb6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:19.284308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a738534697947c8a4c15a4eb32d2072', 'request_id': 'req-bc42e217-60dd-4ed8-814d-f22300615bb6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:19.290294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a738534697947c8a4c15a4eb32d2072', 'request_id': 'req-bc42e217-60dd-4ed8-814d-f22300615bb6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:19.302307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] The token cache is being invalidate because project 4a738534697947c8a4c15a4eb32d2072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:19.302596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:19.303254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bc42e217-60dd-4ed8-814d-f22300615bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:19.307330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1204/2391] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:19 2022] DELETE /identity/v3/projects/4a738534697947c8a4c15a4eb32d2072 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:19.311699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.312215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.312300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.341024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:19.342524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:19.410885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1188/2392] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:19.419316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.419618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.419910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.444432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:19.445787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:19.454150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1205/2393] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:16:19.458907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.469012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884278-1198-4b84-a6ac-9728b59bdc21 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.473704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.474525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.474658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.474890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.475538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:19.476090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.478042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.490176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1189/2394] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:16:19.494693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.505094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62518afb-ec5f-40d8-bfe1-1720c5c119ac None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.513598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.514229 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.514508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.514779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.515433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.517384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.549931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1206/2395] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:19 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:19.554428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.565792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dece3-4025-4576-b51c-618d94829c12 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.636512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.637372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.637886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.638325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.639146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.641485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.667882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1190/2396] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:19 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:19.669400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.680041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d42eb-5a1b-45cc-8576-65787d29a7a4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.684910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.685475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.685995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.685995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.686658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.688264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.699363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1207/2397] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:16:19.706329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.719355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94213b50-7d65-41fd-9f6d-5562ec0c88dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.724465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.725192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.725545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.725830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.727046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.728791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4c2de-b875-4ea2-b402-36df863188be admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.747128 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1191/2398] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:16:19.751579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.763490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8dc5c-4fad-47e9-a486-53e64bb5e95d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:19.769824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:19.770071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.770411 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.770749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] PATH_INFO: `/v3/projects/55431c07117e413fbf32d7050c036e95/users/aba910edd78a4fe08b789eaf46cb2c99/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.789937 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55431c07117e413fbf32d7050c036e95, user_id=aba910edd78a4fe08b789eaf46cb2c99, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:19.792686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:19.807603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1208/2399] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:19 2022] PUT /identity/v3/projects/55431c07117e413fbf32d7050c036e95/users/aba910edd78a4fe08b789eaf46cb2c99/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:19.819516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad341b-dfe6-4731-bd2a-692d2b0a3642 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:19.819516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad341b-dfe6-4731-bd2a-692d2b0a3642 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:19.819516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad341b-dfe6-4731-bd2a-692d2b0a3642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:19.852726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68ad341b-dfe6-4731-bd2a-692d2b0a3642 None None] MFA Rules not processed for user `aba910edd78a4fe08b789eaf46cb2c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:19.854008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68ad341b-dfe6-4731-bd2a-692d2b0a3642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:19.866170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:19.956175 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1192/2400] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:19.968625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:19.970511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:20.036890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dcd519-07ec-4d58-b30d-47654a891613 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:20.041273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:20.041901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:20.042180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:20.042455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:20.051027 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:20.052462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb65217-f68c-45c5-b733-7eee308df271 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:20.062545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1209/2401] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:20.953062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:20.958156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:20.958505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:20.982320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] MFA Rules not processed for user `db4f0928502a43b490de01afc587a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:20.983544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:21.060782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1193/2402] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:25.438016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:25.438566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:25.438566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:25.458771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:25.460104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:25.473145 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:25.521977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1210/2403] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:16:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:25.535390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:25.538057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:25.617313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9e05d-5089-4c7b-9ea2-87bbd6b3d7e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:25.622687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:25.623310 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:25.623766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:25.624048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:25.637362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:25.637362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71a584-f197-44f5-980f-5c2030e4fba2 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:25.644795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1194/2404] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:16:35.515075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=18 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:35.640955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=79 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:43.487413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:43.562322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee891da-98b5-42ca-a65e-e6f51afaf306 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:43.567375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:43.568149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:43.568495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:43.569153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] PATH_INFO: `/v3/users/aba910edd78a4fe08b789eaf46cb2c99` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:43.570538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba910edd78a4fe08b789eaf46cb2c99)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:43.572687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:43.635252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba910edd78a4fe08b789eaf46cb2c99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:43.635987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba910edd78a4fe08b789eaf46cb2c99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:43.644137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba910edd78a4fe08b789eaf46cb2c99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:43.645096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba910edd78a4fe08b789eaf46cb2c99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:43.656006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba910edd78a4fe08b789eaf46cb2c99'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:43.698955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1211/2405] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:43 2022] DELETE /identity/v3/users/aba910edd78a4fe08b789eaf46cb2c99 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:43.699701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:43.700144 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:43.700234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:43.766182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:43.768462 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:43.873796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1195/2406] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:43.884562 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:43.886224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:43.974625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccdd42-4c4f-4d19-ae9e-82fc445722de admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:43.978475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:43.979119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:43.979400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:43.979668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:43.990468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:43.991167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f878ba-f0f4-496e-9c9a-ecf795add68c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.002421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1212/2407] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:16:44.315294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.316738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:44.346879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a94b23-ab1d-4fe5-b1b5-34d188a0ab29 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.353517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.354159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.354440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.354715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] PATH_INFO: `/v3/projects/55431c07117e413fbf32d7050c036e95` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.366035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55431c07117e413fbf32d7050c036e95)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.368251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.456972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55431c07117e413fbf32d7050c036e95', 'request_id': 'req-93eb517a-5070-4fec-b050-ea45764e3029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:44.467862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55431c07117e413fbf32d7050c036e95', 'request_id': 'req-93eb517a-5070-4fec-b050-ea45764e3029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:44.478399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55431c07117e413fbf32d7050c036e95', 'request_id': 'req-93eb517a-5070-4fec-b050-ea45764e3029'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:44.500195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] The token cache is being invalidate because project 55431c07117e413fbf32d7050c036e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:44.500877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:44.501699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93eb517a-5070-4fec-b050-ea45764e3029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:44.510869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.512621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.513087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.516169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1196/2408] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:44 2022] DELETE /identity/v3/projects/55431c07117e413fbf32d7050c036e95 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:44.550223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:44.551530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:44.577274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:44.635507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1213/2409] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:44.643431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.643717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.643997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.667330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:44.668563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:44.680572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1197/2410] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:44.681541 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.691491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d33839-e317-492e-a21e-a94e84aa1b1f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.696699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.697330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.697635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.697907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.698547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:44.698913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.700526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.712736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1214/2411] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:16:44.717017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.726284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6adaf-23da-4406-9601-9119706005d2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.731038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.731645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.731924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.732202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.732830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.734761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.772014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1198/2412] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:44 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:44.776096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.786974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de47b-2ecf-4f6c-9453-ea71710124be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.845803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.846406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.846686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.846982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.847641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.849574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.875233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1215/2413] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:44 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:16:44.876538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.887707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e3d52-38b8-4379-8320-bc30edb24214 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.892484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.893187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.893479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.893750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.894433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.896142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.908057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1199/2414] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:44.911949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.922488 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc014-1d69-4d1e-b13c-4e551225e3aa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.927554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.928738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.929026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.929994 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.930824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.932880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf0160-595c-4e94-b644-d9aaad036fe1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:44.943668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1216/2415] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:44.948380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:44.961700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d99780-b1f7-4d81-9b38-f18228bb2a66 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:44.967465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:44.968121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:44.968401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:44.968674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] PATH_INFO: `/v3/projects/85ace92216f14b179d4fb312da149148/users/32e3317216a84f1391b7a06f36b1e402/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:44.983991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ace92216f14b179d4fb312da149148, user_id=32e3317216a84f1391b7a06f36b1e402, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:44.986678 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:45.000583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1200/2416] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:44 2022] PUT /identity/v3/projects/85ace92216f14b179d4fb312da149148/users/32e3317216a84f1391b7a06f36b1e402/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:45.012842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde43e4-c47e-404f-91eb-3bc8fbcc293d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:45.013139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde43e4-c47e-404f-91eb-3bc8fbcc293d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:45.013431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde43e4-c47e-404f-91eb-3bc8fbcc293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:45.042068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9bde43e4-c47e-404f-91eb-3bc8fbcc293d None None] MFA Rules not processed for user `32e3317216a84f1391b7a06f36b1e402`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:45.044142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9bde43e4-c47e-404f-91eb-3bc8fbcc293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:45.109461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1217/2417] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:45.121481 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:45.123339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:45.145715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:45.186531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d14cd8-5d79-47c2-9cec-b507b37926b2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:45.190528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:45.191311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:45.191687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:45.192061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:45.201791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:45.203268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:45.212936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1201/2418] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:48.276442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:48.276926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:48.277034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:48.301909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:48.303118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:48.316009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:48.365253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1218/2419] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:16:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:48.392804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:48.394268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:48.470032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ea5c8-0648-4345-9820-98da10dd50e7 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:48.476272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:48.477534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:48.477824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:48.478101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:48.497786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:48.499807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107106de-a1cc-48f1-b7cd-f333e11a8a5d service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:48.518758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1202/2420] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:16:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:16:53.306748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:53.308313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:53.545063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6470e77-d769-452f-8aaa-5951107718f6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:53.557604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:53.558428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:53.558739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:53.559004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] PATH_INFO: `/v3/users/fa83947023284a13af9fac67cb944db2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:53.564408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa83947023284a13af9fac67cb944db2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:53.567618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:53.759536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa83947023284a13af9fac67cb944db2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:53.767471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa83947023284a13af9fac67cb944db2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:53.800242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa83947023284a13af9fac67cb944db2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:53.806428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa83947023284a13af9fac67cb944db2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:53.823829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa83947023284a13af9fac67cb944db2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:53.862572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1219/2421] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:53 2022] DELETE /identity/v3/users/fa83947023284a13af9fac67cb944db2 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:53.864335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:53.864846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:53.865081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:53.889894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:53.891255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:53.981295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1203/2422] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:53.995859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:53.998008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:54.066690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:54.081692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7dc108-21be-46ec-b8bb-16ec1ead0fd4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:54.087361 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:54.088342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:54.088983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:54.089655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:54.101005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:54.102808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b01a6-5d54-4984-b315-dd79c063c1df service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:54.113620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1220/2423] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:54.430063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:54.431931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:54.489328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb2d1c-0365-4ffe-9b67-30a8e49885e5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:54.517937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:54.518690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:54.519270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:54.519270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] PATH_INFO: `/v3/projects/eb28351e15394afa975d935eea447fef` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:54.520493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb28351e15394afa975d935eea447fef)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:54.523997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:54.596490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:54.599338 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb28351e15394afa975d935eea447fef', 'request_id': 'req-02fb0fa7-177d-44f9-a354-bc4171ad9768'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:54.606116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb28351e15394afa975d935eea447fef', 'request_id': 'req-02fb0fa7-177d-44f9-a354-bc4171ad9768'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:54.626543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb28351e15394afa975d935eea447fef', 'request_id': 'req-02fb0fa7-177d-44f9-a354-bc4171ad9768'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:54.664250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] The token cache is being invalidate because project eb28351e15394afa975d935eea447fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:54.664579 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:54.665270 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-02fb0fa7-177d-44f9-a354-bc4171ad9768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:54.667300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1204/2424] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:54 2022] DELETE /identity/v3/projects/eb28351e15394afa975d935eea447fef => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:54.672524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:54.673185 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:54.673494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:54.726173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:54.727564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:54.906846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1221/2425] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:16:54.915137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:54.916062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:54.916499 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:54.945604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:54.947818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:54.962113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1205/2426] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:16:54.966287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:54.998299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a933de0-9d57-4fbf-8517-57acbd7f5f27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.007043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.009565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.010299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.010912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.012624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:55.012955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.015490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.047160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.068792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1222/2427] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:16:55.069953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94398a-1e7a-48aa-8c30-6d05b98f801b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.077952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.078574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.078848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.079118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.079752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.084498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.151179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.154048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1206/2428] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:55 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:55.174020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecc211-e4ab-4add-a2e5-3666f7904700 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.309378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.310449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.310735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.311527 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.312252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.316718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.366084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1223/2429] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:55 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:16:55.370714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.383741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2f37-83b5-4d6f-8f9d-5399c9234f51 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.395426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.396047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.396324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.396594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.397336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.399487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.417737 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1207/2430] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:55.421493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.443924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bba8-7505-401a-92df-8c6b8f633f3a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.459462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.460763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.463244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.463244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.463244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.468652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368886e-f284-45c2-9db6-f86a355aaa56 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.483016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1224/2431] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:16:55.487498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.499191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba144c-fb02-4d29-88e3-9136026a2d87 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.504848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.505857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.506148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.506433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] PATH_INFO: `/v3/projects/b2adf3b130ec4fd78c263f0d730c33cd/users/6bf5b819cf8d49f187edadaa8c174f25/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.525189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2adf3b130ec4fd78c263f0d730c33cd, user_id=6bf5b819cf8d49f187edadaa8c174f25, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.528048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.553479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1208/2432] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:55 2022] PUT /identity/v3/projects/b2adf3b130ec4fd78c263f0d730c33cd/users/6bf5b819cf8d49f187edadaa8c174f25/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:55.556344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.556695 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.557087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.624007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] MFA Rules not processed for user `6bf5b819cf8d49f187edadaa8c174f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:55.625320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:55.719213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:55.800496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.802391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:55.823581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1225/2433] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:16:55.839314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:55.841257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:55.942620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfa4ca-076f-41c6-b36b-120dc63d6b7b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.961327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.961984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.962212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222fc4-0c60-4539-94e5-ecb1cd8b7a09 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:55.962678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.962678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.968458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:55.969101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:55.969414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:55.969752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] PATH_INFO: `/v3/users/e61c3bf2af8745dca92d43d3f78bda06` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:55.971893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61c3bf2af8745dca92d43d3f78bda06)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.981859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:55.984598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:55.990364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.001807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1226/2434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:56.036299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:56.057311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61c3bf2af8745dca92d43d3f78bda06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.069446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93d4b7-fe26-41e3-8709-fd100c81c7c7 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:56.076833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61c3bf2af8745dca92d43d3f78bda06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.088316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:56.088575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.088754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.089035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] PATH_INFO: `/v3/users/6bf5b819cf8d49f187edadaa8c174f25` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.092777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf5b819cf8d49f187edadaa8c174f25)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:56.093070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61c3bf2af8745dca92d43d3f78bda06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.094106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61c3bf2af8745dca92d43d3f78bda06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.095639 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.117198 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:56.128616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61c3bf2af8745dca92d43d3f78bda06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.145655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1209/2435] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:55 2022] DELETE /identity/v3/users/e61c3bf2af8745dca92d43d3f78bda06 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:16:56.145655 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:56.146472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.220914 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf5b819cf8d49f187edadaa8c174f25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.221984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf5b819cf8d49f187edadaa8c174f25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.240189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf5b819cf8d49f187edadaa8c174f25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.241218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf5b819cf8d49f187edadaa8c174f25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.261234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c0bcbf7-a389-4257-863f-7c8ab38be5e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf5b819cf8d49f187edadaa8c174f25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.286892 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1227/2436] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:56 2022] DELETE /identity/v3/users/6bf5b819cf8d49f187edadaa8c174f25 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:56.288384 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d868f-737f-4c46-94b2-d46577498f3f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.288690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d868f-737f-4c46-94b2-d46577498f3f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.289157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d868f-737f-4c46-94b2-d46577498f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.306102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518d9c3-790a-4e50-986c-9e18a40be776 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:56.324749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ac3d868f-737f-4c46-94b2-d46577498f3f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:56.327059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ac3d868f-737f-4c46-94b2-d46577498f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.411494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:56.412133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.412409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.412684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] PATH_INFO: `/v3/users/32e3317216a84f1391b7a06f36b1e402` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.415952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e3317216a84f1391b7a06f36b1e402)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:56.418773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.420023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:56.445665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1228/2437] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:56.451799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.452099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.452369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.477719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:56.479295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.489186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1229/2438] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:56.490048 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:56.492322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.542311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e3317216a84f1391b7a06f36b1e402'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.557493 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e3317216a84f1391b7a06f36b1e402'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.574773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e3317216a84f1391b7a06f36b1e402'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.575658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e3317216a84f1391b7a06f36b1e402'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.599982 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2492a-1eed-4fd4-be48-5beb1ee66fd4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:56.604795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:56.605538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.605741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.605984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.609244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.609908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e3317216a84f1391b7a06f36b1e402'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:56.626961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1210/2439] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:16:56 2022] DELETE /identity/v3/users/32e3317216a84f1391b7a06f36b1e402 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:56.627168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:56.627273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:56.627381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a892410-ef7f-4d06-9223-07b0ca92de22 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.636971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.640750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1230/2440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:56.642473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.643121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.643500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.667076 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:56.668737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.715104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:56.793798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1231/2441] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:56.815158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:56.824755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e945-ec30-4be4-a4b6-7eb62aa1d70d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:56.834061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:56.834986 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.837049 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.838070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.840634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667acdf-dc75-4d91-a21f-942a91b6ec27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:56.842364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:56.850637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:56.852968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:56.854008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:56.854538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:56.871677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:56.873623 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.878036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:56.879613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:56.904571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1211/2442] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:16:56.906256 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1232/2443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:16:57.221149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:57.222797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:57.239385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aab3c4-2ca0-4460-976c-ee84b7a936a5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:57.265952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:57.266665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:57.266945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:57.267545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] PATH_INFO: `/v3/projects/5959f07025da467d8de0ac85b80be04c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:57.268639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5959f07025da467d8de0ac85b80be04c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:57.271102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:57.378472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5959f07025da467d8de0ac85b80be04c', 'request_id': 'req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.390042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5959f07025da467d8de0ac85b80be04c', 'request_id': 'req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.406366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5959f07025da467d8de0ac85b80be04c', 'request_id': 'req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.432346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] The token cache is being invalidate because project 5959f07025da467d8de0ac85b80be04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:57.432690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.433354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.435187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1212/2444] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:57 2022] DELETE /identity/v3/projects/5959f07025da467d8de0ac85b80be04c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:57.558244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:57.562184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:57.563453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:57.570340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:57.631579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927fa99-a6bd-4f5a-b9e8-87e30499137d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:57.761267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fffb3-2265-43c9-a05d-2cfc6501fbc8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:57.767973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:57.768626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:57.769368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:57.769368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] PATH_INFO: `/v3/projects/b2adf3b130ec4fd78c263f0d730c33cd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:57.770987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:57.771216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2adf3b130ec4fd78c263f0d730c33cd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:57.772110 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:57.772577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:57.773574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] PATH_INFO: `/v3/projects/85ace92216f14b179d4fb312da149148` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:57.774491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:57.776031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ace92216f14b179d4fb312da149148)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:57.778212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:57.812345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:57.848663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ace92216f14b179d4fb312da149148', 'request_id': 'req-8a2f2119-d43e-4c8d-84e9-b57024a209ea'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.857224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:57.862227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2adf3b130ec4fd78c263f0d730c33cd', 'request_id': 'req-4fe15240-871d-4f82-a2cf-c0c1c998ec01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.863707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ace92216f14b179d4fb312da149148', 'request_id': 'req-8a2f2119-d43e-4c8d-84e9-b57024a209ea'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.875725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ace92216f14b179d4fb312da149148', 'request_id': 'req-8a2f2119-d43e-4c8d-84e9-b57024a209ea'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.885497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2adf3b130ec4fd78c263f0d730c33cd', 'request_id': 'req-4fe15240-871d-4f82-a2cf-c0c1c998ec01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.893209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] The token cache is being invalidate because project 85ace92216f14b179d4fb312da149148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:57.893850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.894856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8a2f2119-d43e-4c8d-84e9-b57024a209ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.897050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1233/2445] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:57 2022] DELETE /identity/v3/projects/85ace92216f14b179d4fb312da149148 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:57.903390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2adf3b130ec4fd78c263f0d730c33cd', 'request_id': 'req-4fe15240-871d-4f82-a2cf-c0c1c998ec01'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.935006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] The token cache is being invalidate because project b2adf3b130ec4fd78c263f0d730c33cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:16:57.935646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.937491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fe15240-871d-4f82-a2cf-c0c1c998ec01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:16:57.948318 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1213/2446] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:16:57 2022] DELETE /identity/v3/projects/b2adf3b130ec4fd78c263f0d730c33cd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:57.954190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:57.954520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:57.954776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:57.978015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:57.979326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:58.060460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1234/2447] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:58.071886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.072445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.072881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.118271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:58.130842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:58.145037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1214/2448] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:58.146098 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.157454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66012dc-0858-4a24-8408-bf1a33764a77 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.164911 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.165949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.166296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.166637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.167363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:16:58.167733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.169598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.187701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1235/2449] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:16:58.190146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.206696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24150c5a-fb8d-46c0-b5c5-3d22fa98a649 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.216589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.217233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.217528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.217800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.218437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.221248 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.262651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1215/2450] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:16:58 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:16:58.263595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.284585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2775f-77e5-4c48-9594-b319887376ab admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.423327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.424847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.425430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.426426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.433941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.438220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.484202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1236/2451] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:16:58 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:16:58.488832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.504381 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716489-d7f3-4896-8f0d-172706cbc24b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.511512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.512136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.513036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.513453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.514506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.517356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.539988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1216/2452] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:16:58.540559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.551251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820c2d-8983-4e99-a5f2-a9a7c9d48517 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.557071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.557751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.558022 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.558291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.559101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.560535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fa4fb-485e-44a8-8e0e-a63d0d691883 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.576606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1237/2453] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:16:58.577797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.604296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665610f6-3bc6-45d2-a475-aa7c99196775 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.609412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.610715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.611251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.611590 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] PATH_INFO: `/v3/projects/7170a1232ab14cd4aff705e91364d5c1/users/c6a74f33e6e941ed8c3f7850bdf29bb0/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.647268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7170a1232ab14cd4aff705e91364d5c1, user_id=c6a74f33e6e941ed8c3f7850bdf29bb0, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.650933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.673911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1217/2454] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:16:58 2022] PUT /identity/v3/projects/7170a1232ab14cd4aff705e91364d5c1/users/c6a74f33e6e941ed8c3f7850bdf29bb0/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:16:58.679951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85274998-d6d7-40d3-89c3-a6a0d2f0feb6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.680360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85274998-d6d7-40d3-89c3-a6a0d2f0feb6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.681005 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85274998-d6d7-40d3-89c3-a6a0d2f0feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.708246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-85274998-d6d7-40d3-89c3-a6a0d2f0feb6 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:16:58.709508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-85274998-d6d7-40d3-89c3-a6a0d2f0feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:58.730700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:16:58.850478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1238/2455] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:16:58.861448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:16:58.863358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:16:58.953060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58af34-4150-4ff8-a22f-7c00e6550c45 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:16:58.957037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:16:58.957798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:16:58.958145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:16:58.958483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:16:58.972173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:16:58.973781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:16:58.986703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1218/2456] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:00.056051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98148e0-7a8b-42db-879f-165052168760 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:00.056915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98148e0-7a8b-42db-879f-165052168760 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:00.057406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98148e0-7a8b-42db-879f-165052168760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:00.112608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d98148e0-7a8b-42db-879f-165052168760 None None] MFA Rules not processed for user `c6a74f33e6e941ed8c3f7850bdf29bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:00.113990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d98148e0-7a8b-42db-879f-165052168760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:00.239157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1239/2457] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:17:00.250054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:00.252196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:00.335228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb78c5e-7698-427b-bf96-4cda9a3542ee service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:00.342958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:00.343957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:00.344384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:00.345155 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:00.362008 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:00.363619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:00.375594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1219/2458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:17:00.547091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e923c99-991d-4095-97a3-77f668159ae6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:00.547395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e923c99-991d-4095-97a3-77f668159ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:00.547651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e923c99-991d-4095-97a3-77f668159ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:00.573509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3e923c99-991d-4095-97a3-77f668159ae6 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:00.575281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e923c99-991d-4095-97a3-77f668159ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:00.671588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1240/2459] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:17:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:00.686447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:00.690134 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:00.742929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:00.835903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419b39-ed98-4941-a8a6-c84224f48650 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:00.841227 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:00.842036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:00.842319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:00.842716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:00.877651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:00.879637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9771-83b3-45d2-9cc5-83103630ca36 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:00.897831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1220/2460] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:17:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:17:02.027119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292f19-5ee6-49e4-a088-434d73fd9d26 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:02.028315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292f19-5ee6-49e4-a088-434d73fd9d26 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:02.028837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb292f19-5ee6-49e4-a088-434d73fd9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:02.072062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cb292f19-5ee6-49e4-a088-434d73fd9d26 None None] MFA Rules not processed for user `c6a74f33e6e941ed8c3f7850bdf29bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:02.073543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb292f19-5ee6-49e4-a088-434d73fd9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:02.092635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1241/2461] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:07.530470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:07.532824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:07.533323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:07.536454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1221/2462] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:17:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:17:07.587280 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778249b1-f480-44c5-bd3e-5fb416d0b83e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:07.589824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778249b1-f480-44c5-bd3e-5fb416d0b83e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:07.590364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778249b1-f480-44c5-bd3e-5fb416d0b83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:07.658947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-778249b1-f480-44c5-bd3e-5fb416d0b83e None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:07.670570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-778249b1-f480-44c5-bd3e-5fb416d0b83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:07.778731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:07.873200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1242/2463] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:17:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:07.885987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:07.887831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:07.994459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7356b-010e-4e60-8f4f-428ef446290a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:08.002029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a9b0f324af44c0f8827b31befa6939b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:08.002647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.003083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.003367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.016212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:08.017718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7239291-63fb-4912-a007-50b975db6bf9 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:08.032239 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1222/2464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:17:08.166448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea3e8-968a-4f2d-ad7b-eb061a6553bb None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.166745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea3e8-968a-4f2d-ad7b-eb061a6553bb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.167007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ea3e8-968a-4f2d-ad7b-eb061a6553bb None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.171586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1243/2465] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:17:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:17:08.181522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234432-993c-4b05-b33b-3cee24c5a33c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.181882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234432-993c-4b05-b33b-3cee24c5a33c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.182084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a234432-993c-4b05-b33b-3cee24c5a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.203755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3a234432-993c-4b05-b33b-3cee24c5a33c None None] MFA Rules not processed for user `605f796f654b4b98aa29355fd4109aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:08.205040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a234432-993c-4b05-b33b-3cee24c5a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:08.218611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1223/2466] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:17:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:08.224158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea3e8-968a-4f2d-ad7b-eb061a6553bb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:08.237310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ea3e8-968a-4f2d-ad7b-eb061a6553bb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:08.243299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:08.243993 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.244337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.244984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.254358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:08.256390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1c626-770b-4474-bc38-5e83918322f4 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:08.266612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1244/2467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:17:08.409673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.410284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.411035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.414107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1224/2468] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:17:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 07:17:08.419846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.421490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.421985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.483738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] MFA Rules not processed for user `3d44b8c931e047408a2d9d4baa490806`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:08.487433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:08.520294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1245/2469] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:17:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:08.535816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:08.539583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:08.565614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836f28d-03ca-4b65-aa38-8621c1fd1faa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:08.571627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:08.571984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:08.572320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:08.572595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:08.583967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:08.585797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aac780-cded-4b05-9158-2b6676dbfb64 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:08.609707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1225/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:10.493085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:10.506310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d222dec-928f-43ff-af9a-1a96b3f6d5fa None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:10.511472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:10.512391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:10.512905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:10.513439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:10.515474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:10.604024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:10.605751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:10.625591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1246/2471] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:15.154132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:15.156274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:15.156828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:15.183228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:15.184858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:15.200100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1226/2472] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Sep 1 07:17:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:15.220306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:15.232997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c261-d7e3-4033-ad09-58ff36f26b64 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:15.238608 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:15.239412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:15.239845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:15.240289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:15.250171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:15.252338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1a18c-4a1e-4ef4-a383-a21bd3f777dd service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:15.264830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1247/2473] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:17:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:17:17.566350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:17.568576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:17.759528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3ef5-b8b0-44b0-b49e-751a6493e3dd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:17.775484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:17.776383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:17.776825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:17.778130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:17.783391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:17.821010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:17.825332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:17.852094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1227/2474] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:21.152458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb42de-074c-4620-9119-c8e99f336036 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:21.152979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb42de-074c-4620-9119-c8e99f336036 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:21.153073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb42de-074c-4620-9119-c8e99f336036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:21.172895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-13fb42de-074c-4620-9119-c8e99f336036 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:21.174268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-13fb42de-074c-4620-9119-c8e99f336036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:21.182961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1248/2475] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 07:17:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:21.202504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:21.215984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d20ddd-d12f-456f-9f06-c99e7402329f service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:21.220992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:21.221629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:21.221911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:21.222188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:21.231188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:21.232776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600fc99-e152-4bb1-96ee-1eb3f197c75f service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:21.236245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:21.243194 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1228/2476] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:17:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:17:23.601684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:23.602603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:23.603050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:23.604838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1249/2477] 10.222.0.46 () {54 vars in 921 bytes} [Thu Sep 1 07:17:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 07:17:23.636982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:23.638086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:23.638595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:23.662977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] MFA Rules not processed for user `25fbbb7831ed429cb38cd7918e1923ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:23.665368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:23.676557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1229/2478] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Sep 1 07:17:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:23.690327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:23.692293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:23.695370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:23.697936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:23.728971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811b9cf-1ba8-4d9c-8d38-efb808c8a549 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:23.732757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce4856-61f2-4ba8-8a12-79fd733c5766 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:23.736532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:23.737725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:23.738352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:23.738791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:23.747799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:23.749267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:23.749651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:23.750152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] PATH_INFO: `/v3/users/1cc519da0b5d40268b29bb1bef89049b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:23.750748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:23.751491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc519da0b5d40268b29bb1bef89049b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:23.752636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:23.753856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:23.771675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1230/2479] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:23.823700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc519da0b5d40268b29bb1bef89049b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:23.824409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc519da0b5d40268b29bb1bef89049b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:23.835405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc519da0b5d40268b29bb1bef89049b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:23.837381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc519da0b5d40268b29bb1bef89049b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:23.851999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e642997-d164-40a0-8f87-cebae26176ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc519da0b5d40268b29bb1bef89049b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:23.874619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1250/2480] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:17:23 2022] DELETE /identity/v3/users/1cc519da0b5d40268b29bb1bef89049b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:17:24.135119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.146032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14c72-7d45-4b92-8e53-63cb6427f957 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.260675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.261763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.262264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.262760 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] PATH_INFO: `/v3/projects/7d030bb54a7c4e489734b7c86b40b464` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.264201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d030bb54a7c4e489734b7c86b40b464)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.266379 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.356006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d030bb54a7c4e489734b7c86b40b464', 'request_id': 'req-5eb00f93-6af0-46b8-96b6-ce960620c812'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:24.362853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d030bb54a7c4e489734b7c86b40b464', 'request_id': 'req-5eb00f93-6af0-46b8-96b6-ce960620c812'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:24.369736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d030bb54a7c4e489734b7c86b40b464', 'request_id': 'req-5eb00f93-6af0-46b8-96b6-ce960620c812'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:24.384544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] The token cache is being invalidate because project 7d030bb54a7c4e489734b7c86b40b464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:17:24.384939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:24.385701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5eb00f93-6af0-46b8-96b6-ce960620c812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:24.387649 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1231/2481] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:17:24 2022] DELETE /identity/v3/projects/7d030bb54a7c4e489734b7c86b40b464 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:17:24.392686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.392975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.393282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.402053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:24.431592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:24.432837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:24.503219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1251/2482] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:24.511416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.511696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.511969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.535899 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:24.537199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:24.548953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1232/2483] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:24.550153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.560284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d2b4-4d0b-4ea9-aa45-a96f8077e782 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.565259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.565878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.566288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.566568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.567219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:17:24.567525 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.569163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.586369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1252/2484] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:17:24.587060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.596540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8b568-7096-456b-a78b-9ce04663e76e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.602123 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.602725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.603000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.603268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.603896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.605850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.634192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1233/2485] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:17:24 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:17:24.638552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.650875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1055109-a0f9-480c-a528-df66685d6cb2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.714989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.715714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.716068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.716407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.717134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.719179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.759154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1253/2486] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:17:24 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:17:24.763268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.773738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f088e6-ea4d-4bb1-ae42-c9fc4e8a081e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.781733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.781733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.781733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.781733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.782210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.783412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.794551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1234/2487] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:17:24.799310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.810504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dadc13-88ae-41b3-8b2f-90dc7e8163e2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.815396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.816003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.816305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.816586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.817212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.818838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23ff25-2043-43d6-b811-932ec17fce4b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.831330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1254/2488] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:17:24.835910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:24.846435 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f2191-4655-4ec4-932c-eec3faf78665 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:24.851821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:24.852480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.852763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.853062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] PATH_INFO: `/v3/projects/bd66de387443438a95808d012a7ae2e0/users/aa5821e8639b401e844334169c5f44d7/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:24.869596 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd66de387443438a95808d012a7ae2e0, user_id=aa5821e8639b401e844334169c5f44d7, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:24.872652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:24.917168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1235/2489] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:17:24 2022] PUT /identity/v3/projects/bd66de387443438a95808d012a7ae2e0/users/aa5821e8639b401e844334169c5f44d7/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:17:24.970908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:24.971277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:24.971456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:25.002877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] MFA Rules not processed for user `aa5821e8639b401e844334169c5f44d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:25.004121 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:25.065732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:25.077504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1255/2490] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:25.086299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:25.088057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:25.095472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:25.159796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da3a8e-9341-4482-b2f6-de458711b8a7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:25.164554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:25.165355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:25.165726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:25.166059 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:25.178145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:25.180317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15263505-3400-4085-8de3-837873c08f23 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:25.190983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1236/2491] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:27.910727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:27.976680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37108a-141d-4fb7-98d0-6e45a10d18b5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:27.981105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:27.981758 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:27.982034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:27.982306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] PATH_INFO: `/v3/users/aa5821e8639b401e844334169c5f44d7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:27.983421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5821e8639b401e844334169c5f44d7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:27.985377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.033200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5821e8639b401e844334169c5f44d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.033850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5821e8639b401e844334169c5f44d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.041786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5821e8639b401e844334169c5f44d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.042410 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5821e8639b401e844334169c5f44d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.050154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5821e8639b401e844334169c5f44d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.067771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1256/2492] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:17:27 2022] DELETE /identity/v3/users/aa5821e8639b401e844334169c5f44d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:17:28.072465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.072945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.073026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.093751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:28.094955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:28.152856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1237/2493] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:28.162246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.164127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:28.224388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab1146-9f6e-493a-aac2-9964304d2854 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:28.227955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:28.228554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.228831 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.229101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.237753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:28.239168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2052f65-3d29-4b13-ab13-5dee648de034 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.248386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1257/2494] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:28.556412 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.557955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:28.574117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853caaf-f468-4934-b02c-b3826507d86f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:28.578693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:28.579322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.579648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.579856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] PATH_INFO: `/v3/projects/bd66de387443438a95808d012a7ae2e0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.580973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd66de387443438a95808d012a7ae2e0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:28.583202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.629050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd66de387443438a95808d012a7ae2e0', 'request_id': 'req-7e30144a-26ad-441e-92c9-4bf6df3a84f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.635812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd66de387443438a95808d012a7ae2e0', 'request_id': 'req-7e30144a-26ad-441e-92c9-4bf6df3a84f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.641633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd66de387443438a95808d012a7ae2e0', 'request_id': 'req-7e30144a-26ad-441e-92c9-4bf6df3a84f6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.653371 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] The token cache is being invalidate because project bd66de387443438a95808d012a7ae2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:17:28.653664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.654277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7e30144a-26ad-441e-92c9-4bf6df3a84f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:28.655741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1238/2495] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:17:28 2022] DELETE /identity/v3/projects/bd66de387443438a95808d012a7ae2e0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:17:28.660617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.660883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.661200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.680179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:28.681395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:28.694610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:28.743310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1258/2496] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:17:28.751415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.751782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.752129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.773157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:28.774395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:28.782393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1239/2497] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:28.786801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.797870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec03a2-63ae-47ed-9764-09e11ab5fcea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:28.802613 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:28.803226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.803501 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.803770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.804586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:17:28.804708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:28.806350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.824393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.825224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1259/2498] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:17:28.836543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c1a9a-977e-452e-99eb-cc4a8619f977 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:28.843382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:28.844525 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.844812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.845092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.845757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:28.847883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.881526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1240/2499] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:17:28 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:17:28.886321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.897051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e895-edaa-46bc-a252-ccd272d2df2a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:28.957174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:28.957846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:28.958054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:28.958325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:28.958963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:28.960878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:28.981560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1260/2500] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:17:28 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:17:28.985753 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:28.991132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:28.997889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4402cdc-c6d7-4e34-8475-0705a49aa579 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:29.003717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:29.004308 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:29.004583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:29.004854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:29.005558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:29.007173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:29.017256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1241/2501] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:17:29.021284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:29.031173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac24d-dca4-4491-b33e-e382f633959c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:29.036198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:29.036792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:29.037071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:29.037385 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:29.038023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:29.039620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaba0ae-fd8a-43cf-8d34-0f8072b59ee8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:29.051255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1261/2502] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:17:29.055522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:29.065974 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf211723-bcfa-45aa-8799-47b9abbadb36 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:29.070568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:29.071206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:29.071486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:29.071765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] PATH_INFO: `/v3/projects/4d31d57f8f3a4cc49a4b1c422b5e823c/users/c73bba0d125b46bf81b1e7a043e66b09/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:29.086972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d31d57f8f3a4cc49a4b1c422b5e823c, user_id=c73bba0d125b46bf81b1e7a043e66b09, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:29.089731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:29.103861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1242/2503] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:17:29 2022] PUT /identity/v3/projects/4d31d57f8f3a4cc49a4b1c422b5e823c/users/c73bba0d125b46bf81b1e7a043e66b09/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:17:29.114261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:29.114550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:29.114823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:29.143580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] MFA Rules not processed for user `c73bba0d125b46bf81b1e7a043e66b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:29.144731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:29.210382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1262/2504] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:29.221602 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:29.223354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:29.286027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b92bd5-730a-4c46-8de1-9eeae0a9a792 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:29.289992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:29.290804 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:29.291610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:29.292014 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:29.301201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:29.302657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9abefd-70c6-46d0-9032-997e7235db60 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:29.311502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1243/2505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:39.207369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:39.307464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:39.421497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:39.485674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34c0f-e8c5-49c1-bc68-0fb35c88987f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:39.490050 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:39.490680 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:39.490955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:39.491223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] PATH_INFO: `/v3/users/c73bba0d125b46bf81b1e7a043e66b09` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:39.492332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73bba0d125b46bf81b1e7a043e66b09)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:39.494320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:39.540566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73bba0d125b46bf81b1e7a043e66b09'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:39.541242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73bba0d125b46bf81b1e7a043e66b09'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:39.548232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73bba0d125b46bf81b1e7a043e66b09'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:39.548856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73bba0d125b46bf81b1e7a043e66b09'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:39.555845 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73bba0d125b46bf81b1e7a043e66b09'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:39.576570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1263/2506] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:17:39 2022] DELETE /identity/v3/users/c73bba0d125b46bf81b1e7a043e66b09 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:17:39.581211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:39.581501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:39.581777 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:39.604139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:39.605384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:39.667829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1244/2507] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:39.677568 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:39.678980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:39.737379 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311832d-db34-4191-b596-b028f766c6de admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:39.740880 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:39.741521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:39.741799 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:39.742089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:39.751235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:39.752668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce1b19-3c23-4cd9-9412-770956906559 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:39.763189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1264/2508] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:17:40.046527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:40.047988 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:40.061105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e58ea2-d150-417b-a1aa-bfcde73338c7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:40.069610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:40.070696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:40.070977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:40.071305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] PATH_INFO: `/v3/projects/4d31d57f8f3a4cc49a4b1c422b5e823c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:40.072945 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d31d57f8f3a4cc49a4b1c422b5e823c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:40.080475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:40.131464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d31d57f8f3a4cc49a4b1c422b5e823c', 'request_id': 'req-ce1793f1-e48a-4ae5-b409-99bdf9983698'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:40.138998 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d31d57f8f3a4cc49a4b1c422b5e823c', 'request_id': 'req-ce1793f1-e48a-4ae5-b409-99bdf9983698'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:40.148993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d31d57f8f3a4cc49a4b1c422b5e823c', 'request_id': 'req-ce1793f1-e48a-4ae5-b409-99bdf9983698'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:40.161550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] The token cache is being invalidate because project 4d31d57f8f3a4cc49a4b1c422b5e823c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:17:40.161876 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:40.162491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:17:40.164021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1245/2509] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:17:40 2022] DELETE /identity/v3/projects/4d31d57f8f3a4cc49a4b1c422b5e823c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:17:43.272108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:43.272645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:43.272645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:43.295598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:17:43.296985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:43.314727 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:43.355655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1265/2510] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 07:17:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:17:43.368472 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:17:43.369866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:17:43.440240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1793f1-e48a-4ae5-b409-99bdf9983698 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:17:43.445365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:17:43.445973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:17:43.446276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:17:43.446548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:17:43.454465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:17:43.456042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:17:43.464167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1246/2511] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:17:53.348655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=15 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:17:53.460975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:19:05.904520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:05.905935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:05.961102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6da4a2-ee71-44d2-875f-ccabf03c02cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:05.964651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:05.965276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:05.965555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:05.965824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:05.967530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:06.036213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:06.037649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a8b75-40f3-4bdb-a384-5d8fd69c71bc service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:06.048163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1266/2512] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:10.737374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:10.738827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:10.753951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9bb3-806d-4ecf-a2f6-e101e839593d service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:10.759423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:10.760017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:10.760557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:10.760831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:10.762543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:10.820347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:10.822955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:10.834266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1247/2513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:11.008624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:11.009154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:11.009154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:11.028383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] MFA Rules not processed for user `db4f0928502a43b490de01afc587a846`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:19:11.029611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:11.036328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1267/2514] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:19:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:19:11.130515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:11.131938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:11.189651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e2189-b5b1-4a8b-a023-b07863847c7e service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:11.193782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:11.194387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:11.194667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:11.194943 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:11.196684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:11.210780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:11.212200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:11.223004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1248/2515] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:19:14.014947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:14.028332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff7d0-939c-469e-b3fe-622c4a177ede None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:14.033712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:14.034305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:14.034592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:14.034870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:14.036622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:14.094273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:14.095828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:14.104687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1268/2516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:21.218182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=54 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:19:24.100490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:19:33.932254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:33.933826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:33.948439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765f74-f97d-4ad9-92ae-2f3f7f3d84c5 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:33.955911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:33.956785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:33.956785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:33.957135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:33.958972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:33.972839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:33.974447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:33.995565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1249/2517] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:34.102026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:34.112353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729018e6-7192-442e-a115-7939326ec9ac service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:34.116158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:34.116800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:34.117233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:34.117860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:34.119569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:34.131517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:34.132983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:34.155648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1269/2518] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:35.164561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:19:35.188437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e206a5-c6df-4f92-86ae-a5c7645ccae4 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:19:35.194402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:19:35.195548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:19:35.195816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:19:35.196814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:19:35.199296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:19:35.216287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:19:35.218034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:19:35.229792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1250/2519] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:19:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:19:44.138709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:19:45.225286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:21:03.896644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:21:03.912675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb37c-4e11-4688-a95c-33ad630f1b57 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:21:03.918745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:21:03.920462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:21:03.920749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:21:03.921024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:21:03.922830 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:21:03.984798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:21:03.986395 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:21:03.995222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1270/2520] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:21:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:21:13.991030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=28 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:22:00.038941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:22:00.040333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:22:00.052107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52bef3-e051-4bfc-8b70-f533c9234641 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:22:00.055588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:22:00.056169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:22:00.056442 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:22:00.056713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:22:00.058429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:22:00.115526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:22:00.116940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:22:00.126933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1251/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:22:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:22:10.121633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=28 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:22:18.289241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:22:18.290633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:22:18.426359 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a182a9f-4943-4886-a9c8-e08c8097fbad service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:22:18.431446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:22:18.432046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:22:18.432326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:22:18.432598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:22:18.434316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:22:18.445548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:22:18.447105 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:22:18.457253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1271/2522] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:22:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:22:28.452597 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=63 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:24:11.458109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:24:11.510622 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45973805-8b61-4193-984e-3d1f524fb80b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:24:11.516465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:24:11.517068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:24:11.517372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:24:11.517645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:24:11.526186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:24:11.527770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:24:11.554241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1252/2523] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:24:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:24:14.436322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:24:14.448751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125141-25d8-484c-9f08-d19951475ff9 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:24:14.454572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:24:14.455176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:24:14.455454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:24:14.455728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:24:14.464933 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:24:14.466584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:24:14.495930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1272/2524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:24:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:24:21.551661 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=31 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:24:24.493362 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=13 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:25:21.858549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:25:21.877933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53097017-79dd-43fb-a34f-00b51085515a service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:25:21.882180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:25:21.882772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:25:21.883047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:25:21.883314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:25:21.890685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:25:21.892106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:25:21.899992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1253/2525] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:25:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:25:31.896990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:25:35.823617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:25:35.842642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452c632-6a8e-4077-9176-e751e2e52ff0 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:25:35.846228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:25:35.846817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:25:35.847095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:25:35.847365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:25:35.858364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:25:35.859768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:25:35.871307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1273/2526] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:25:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:25:45.866598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:26:02.473659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:26:02.483415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051ae97-7713-4e31-b36f-2fee26bb21ca service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:26:02.486997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:26:02.487578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:26:02.487851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:26:02.488117 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:26:02.496399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:26:02.497834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:26:02.524529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1254/2527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:26:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:26:12.521505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=13 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:27:00.479188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:27:00.488420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57f172-f2ac-4575-9007-dfe90a66926e service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:27:00.492592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:27:00.493209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:27:00.493489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:27:00.493757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:27:00.501846 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:27:00.503253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:27:00.529675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1274/2528] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:27:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:27:10.527073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=13 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:27:26.270728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:27:26.282095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62e2e2-982b-4c56-a59d-b5e95bad04e8 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:27:26.287674 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:27:26.288302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:27:26.288584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:27:26.288859 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:27:26.297385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:27:26.298982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:27:26.309280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1255/2529] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:27:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:27:36.305106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:29:11.839595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:29:11.897007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f085e6-e7fa-408f-a96f-fd29a2051173 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:29:11.900699 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:29:11.901324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:29:11.901607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:29:11.901878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:29:11.903820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:29:11.972453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:29:11.973924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:29:11.982822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1275/2530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:29:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:29:14.922862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:29:14.932964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c065b0-6ce0-41f6-bd0c-9e97f640fe8d service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:29:14.936555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:29:14.937180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:29:14.937465 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:29:14.937741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:29:14.939582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:29:15.015004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:29:15.016426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:29:15.025526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1256/2531] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:29:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:29:21.978646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=53 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:29:25.021524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=33 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:29:56.520090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:29:56.521684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:29:56.587955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c215600-b4d1-4e9d-b2b0-2b63a0834ca7 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:29:56.591543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:29:56.592131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:29:56.592408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:29:56.592679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:29:56.594493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:29:56.674244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:29:56.675658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:29:56.686909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1276/2532] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:29:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:30:06.682218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=64 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:30:24.894431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:30:24.904429 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173e432-6447-4472-8365-1157e11434d3 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:30:24.908009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:30:24.908608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:30:24.908885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:30:24.909190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:30:24.911176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:30:24.920648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:30:24.922113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:30:24.930776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1257/2533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:30:29.182092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:30:29.183743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:30:29.203310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d73c10-3f9b-4b90-8aa3-7f103a8f1853 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:30:29.216905 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:30:29.217565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:30:29.217908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:30:29.218286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:30:29.220817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:30:29.238116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:30:29.239965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:30:29.268922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1277/2534] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:30:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:30:34.596453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:30:34.606147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f5887-edcb-426d-b3bb-b607701e9e00 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:30:34.609997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:30:34.610591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:30:34.610871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:30:34.611140 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:30:34.613005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:30:34.627814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:30:34.629275 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:30:34.640861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1258/2535] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:30:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:30:39.248976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:30:44.635562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:32:00.253465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:32:00.265882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aab7c-dc51-48dc-a0a2-b6ee1c6e6273 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:32:00.271368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:32:00.271973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:32:00.272248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:32:00.272516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:32:00.274344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:32:00.344567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:32:00.346200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:32:00.355071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1278/2536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:32:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:32:10.350980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=33 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:32:28.282867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:32:28.284495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:32:28.396482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d7fa-4fe3-4e64-94d8-a03bb00e4f84 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:32:28.402572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:32:28.403176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:32:28.403456 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:32:28.403725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:32:28.405668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:32:28.417137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:32:28.418735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:32:28.428495 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1259/2537] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:32:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:32:29.388563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:32:29.390510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:32:29.423785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f3a88-6fca-489e-b305-2c0e814c8524 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:32:29.427625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:32:29.428227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:32:29.428507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:32:29.428778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:32:29.430636 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:32:29.499963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:32:29.501427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:32:29.510706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1279/2538] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:32:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:32:38.424290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=52 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:32:39.506464 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=42 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:34:11.721452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:34:11.772909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53909e-0a8c-4662-8159-0a5b2f7d51fc service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:34:11.779077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:34:11.779681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:34:11.779961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:34:11.780233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:34:11.788116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:34:11.789765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:34:11.817045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1260/2539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:34:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:34:13.634003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:34:13.635626 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:34:13.673929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df078c1-c198-4a09-8d02-ab3d515301f2 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:34:13.679559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:34:13.680678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:34:13.680955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:34:13.681258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:34:13.689101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:34:13.690762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:34:13.700193 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1280/2540] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:34:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:34:21.814264 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=31 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:34:23.696047 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=17 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:34:56.526888 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:34:56.536038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e6197-b7c6-42da-899e-98999b750457 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:34:56.539732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:34:56.540361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:34:56.540640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:34:56.540903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:34:56.551788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:34:56.553234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:34:56.565150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1261/2541] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:34:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:35:06.560005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:35:27.875500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:35:27.895596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a37d49-c30a-4601-ae9f-6e45ca30dca7 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:35:27.899178 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:35:27.900412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:35:27.900817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:35:27.901090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:35:27.908374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:35:27.909814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bec823-3e94-47b6-8fdc-5e77010b960c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:35:27.918417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1281/2542] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:35:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:35:34.966639 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:35:34.985306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f10a3-5a2c-41ae-8bdf-c21b1a88df3b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:35:34.988955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:35:34.989585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:35:34.989864 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:35:34.990141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:35:35.001949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:35:35.003402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:35:35.017707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1262/2543] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:35:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:35:37.914301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:35:45.010658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:36:31.360166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:36:31.360683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:36:31.360683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:36:31.396848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] MFA Rules not processed for user `2096dd4d4b6f49fc8572a4bcd9689248`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:36:31.398073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:36:31.414271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1282/2544] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:36:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:36:31.713813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:36:31.722143 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1604a13-544d-4c0a-9bc8-6999b87ccd2f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:36:31.725658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:36:31.726253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:36:31.726522 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:36:31.726787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:36:31.729097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:36:31.738903 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:36:31.740355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:36:31.748829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1263/2545] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:36:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:36:33.205249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:36:33.218753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3844a3-1fa2-4cda-b8a7-77dd790557e3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:36:33.223839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:36:33.224449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:36:33.224725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:36:33.224996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:36:33.226747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:36:33.239197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:36:33.240775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:36:33.251414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1283/2546] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:36:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:36:36.431399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:36:36.447927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0d5f-0658-4c24-afbc-49e220f0fb41 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:36:36.453629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:36:36.454174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:36:36.454449 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:36:36.454716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:36:36.464453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:36:36.466055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:36:36.478915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1264/2547] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:36:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:36:42.131012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:36:42.144072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cd988-0e19-4d46-ad9e-5d6fb2c73f2b service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:36:42.150226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:36:42.150833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:36:42.151107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:36:42.151376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:36:42.153302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:36:42.166366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:36:42.167966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:36:42.178646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1284/2548] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:36:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:36:46.473421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:36:52.173972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=26 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:37:30.262068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:37:30.275831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415a27-4e4a-49c0-a449-e389c7b15899 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:37:30.280977 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:37:30.281633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:37:30.281901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:37:30.282174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:37:30.291277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:37:30.293439 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:37:30.321357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1265/2549] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:37:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:37:31.028720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:37:31.039953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aead1-a2df-4bdb-b485-3d101a3b5a77 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:37:31.044925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:37:31.045579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:37:31.045862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:37:31.046137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:37:31.054552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:37:31.056127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e65a7a-bca0-4d31-9237-950737b608e4 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:37:31.065851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1285/2550] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:37:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:37:40.318497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=13 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:37:41.061679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:30.115929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:30.147869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c8fc3-f116-4a5d-8f85-44d3d3b5db45 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:30.151672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:30.152284 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:30.152561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:30.152879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:30.163111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:30.165035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:30.194296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1266/2551] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:38:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:38:30.333091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:30.333650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:30.333650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:30.429000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] MFA Rules not processed for user `d253c4509e954c11927f3f1f4563a5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:30.430496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:30.536759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1286/2552] 10.222.0.46 () {58 vars in 967 bytes} [Thu Sep 1 07:38:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:30.612621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:30.616714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:30.723394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60b44b-6a90-48f6-9ed1-5d6b558271c7 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:30.728351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6834988c33e94b9cb41df95e4bb1bd88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:30.728949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:30.729268 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:30.729545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:30.737989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:30.739648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:30.748607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1267/2553] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Sep 1 07:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 07:38:31.287173 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:31.301948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcfbce-f375-4279-a8eb-54e2e5dd8a82 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:31.306111 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:31.306952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:31.307424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:31.307855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:31.308078 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:31.311060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:31.321511 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7505c-d225-422f-9678-c747f9941446 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:31.329193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:31.330066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:31.330443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:31.330821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:31.332878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:31.346832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:31.348512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df071f7-0286-447d-8d80-3c0edffd3441 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:31.359955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1268/2554] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:38:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:38:31.413107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:31.414569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ea10a-8b4a-4dbc-9c12-8c160ce8abb8 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:31.433859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1287/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:38:34.415214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.415779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.415779 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.479101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:34.480368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:34.488203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1269/2556] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:34.495995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.496469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.496606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.521862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:34.523157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:34.532092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1288/2557] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:34.536344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.546218 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4a67c-add0-4c75-a264-bcb9d8cb7955 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.550798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.551398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.551677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.551952 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.552595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:38:34.552895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.554685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.566417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1270/2558] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:38:34.570573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.580302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0057e-da67-41fc-b5b9-125cf72d4d0e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.585191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.585823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.586102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.586376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.587000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.588908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.609974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:34.625816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1289/2559] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:38:34 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:38:34.629718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.640093 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989437-817a-4e2c-ae27-fac87f4ba1c2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.665404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:34.696583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.697214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.697496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.697771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.698390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.700294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.721505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1271/2560] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:38:34 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:38:34.725616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.736327 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b61af8-152e-4a92-80a8-e7c46150daa7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.740969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.741607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.741884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.742153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.742789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.744373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.754936 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1290/2561] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:38:34.758985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.769361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c94ae-cc49-4240-b52a-b5fa73b3fe61 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.773995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.774601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.774878 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.775172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.775805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.777453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c992f4-0f79-40bc-85e1-2b2b4bc30dcb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.792929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1272/2562] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:38:34.798837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:34.809046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca774726-3ff8-4aeb-a539-8441381cace9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:34.814535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:34.815198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.815478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.815753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] PATH_INFO: `/v3/projects/bbc5effaf211410eb0b8872c2be553da/users/d3a4f1b15f0442578e65b810edfb4923/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.831068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc5effaf211410eb0b8872c2be553da, user_id=d3a4f1b15f0442578e65b810edfb4923, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:34.833793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:34.846429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1291/2563] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:38:34 2022] PUT /identity/v3/projects/bbc5effaf211410eb0b8872c2be553da/users/d3a4f1b15f0442578e65b810edfb4923/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:38:34.851536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a807f-3393-4b2c-995a-d7eec84355c0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:34.851822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a807f-3393-4b2c-995a-d7eec84355c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:34.852120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a807f-3393-4b2c-995a-d7eec84355c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:34.871263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d2a807f-3393-4b2c-995a-d7eec84355c0 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:34.872470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d2a807f-3393-4b2c-995a-d7eec84355c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:34.933405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1273/2564] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:34.943819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:35.001389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6e494-9a29-4afd-92f0-e84d84f6f627 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:35.005204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:35.005770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:35.006046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:35.006314 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:35.015795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:35.017531 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:35.027719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1292/2565] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:38:39.007454 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.007995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.007995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.036273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] MFA Rules not processed for user `d3a4f1b15f0442578e65b810edfb4923`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:39.037533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:39.105708 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1274/2566] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:39.110409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.120293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a80760-e48f-42ff-b8e6-817d044afb4d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.127986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.128616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.128881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.129170 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.129826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.131738 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.169168 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1293/2567] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:38:39 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:38:39.171387 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.195137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c6c1-b60d-4c74-ad8a-7939d710afca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.235279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:39.262137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.262845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.263781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.264171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.264874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.267217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.308046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1275/2568] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:38:39 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:38:39.308963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.331234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621947a7-55b5-452f-8a17-8a69da87f5c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.335916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.336518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.336792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.337059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.337909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.339504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.350481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1294/2569] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:38:39.357667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.378592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207c98f-9713-4a01-a737-f6a0de1309c3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.383840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.384606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.385009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.385350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] PATH_INFO: `/v3/projects/09bdfceb1dd84035a3cf16bc3c261292/users/b09bb0c8090c4d4088da847e972c2aa2/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.401884 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09bdfceb1dd84035a3cf16bc3c261292, user_id=b09bb0c8090c4d4088da847e972c2aa2, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.404763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.419168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1276/2570] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:38:39 2022] PUT /identity/v3/projects/09bdfceb1dd84035a3cf16bc3c261292/users/b09bb0c8090c4d4088da847e972c2aa2/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:38:39.423640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.435088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d0106-8058-4c80-a45b-5d91629c2c87 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.494611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.495221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.495497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.495761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.496396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.498101 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b857d8c-ecb7-4824-89b1-a0d59811e0ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.511884 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1295/2571] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:38:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:38:39.513309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:39.524563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ea86-363c-4cb3-b7d2-c429d07cd114 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:39.530086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:39.530832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:39.531112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:39.531416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] PATH_INFO: `/v3/projects/09bdfceb1dd84035a3cf16bc3c261292/users/b09bb0c8090c4d4088da847e972c2aa2/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:39.533403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09bdfceb1dd84035a3cf16bc3c261292, user_id=b09bb0c8090c4d4088da847e972c2aa2, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:39.536062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:39.548003 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1277/2572] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:38:39 2022] PUT /identity/v3/projects/09bdfceb1dd84035a3cf16bc3c261292/users/b09bb0c8090c4d4088da847e972c2aa2/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:38:43.796542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30410-c2c3-42c1-a2c5-e3f37c4052e6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:43.797133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30410-c2c3-42c1-a2c5-e3f37c4052e6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:43.797133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b30410-c2c3-42c1-a2c5-e3f37c4052e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:43.837749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-65b30410-c2c3-42c1-a2c5-e3f37c4052e6 None None] MFA Rules not processed for user `b09bb0c8090c4d4088da847e972c2aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:43.842272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-65b30410-c2c3-42c1-a2c5-e3f37c4052e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:43.854104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:43.975768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1296/2573] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:43.993434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:38:44.087303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770611c-745e-4939-aecf-97b4aaa93013 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:38:44.093246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:38:44.093874 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:44.094156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:44.094431 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:44.154893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:38:44.156486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:38:44.165896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1278/2574] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:38:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:38:45.164809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:38:45.165312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:38:45.165648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:38:45.185589 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] MFA Rules not processed for user `d3a4f1b15f0442578e65b810edfb4923`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:38:45.186814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:38:45.194452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1297/2575] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:38:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:38:54.161578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:38:55.180664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=27 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:40:30.669416 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:40:30.725188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946bce36-9a01-4ce1-b1c4-3b484ac04664 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:40:30.730733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:40:30.731523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:40:30.731914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:40:30.732224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:40:30.734397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:40:30.810114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:40:30.811613 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:40:30.821904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1279/2576] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:40:30.882759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:40:30.884778 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:40:30.908547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:40:30.917304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf2b61-eaae-4d90-98be-424d79c9858e service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:40:30.920821 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:40:30.921446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:40:30.921722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:40:30.921993 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:40:30.924102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:40:30.966396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306bb27-3acc-49d9-9ecf-57bc8b5810c8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:40:30.971017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:40:30.971919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:40:30.972199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:40:30.972825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:40:30.975653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:40:30.986510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:40:30.987826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:40:30.996449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1298/2577] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:40:31.001333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:40:31.003098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:40:31.014447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1280/2578] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:40:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:40:40.992341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=31 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:40:41.010079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:42:31.380258 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:42:31.381864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:42:31.423763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48d660-368e-42a5-a3e5-fdda4d066580 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:42:31.427287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:42:31.427874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:42:31.428148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:42:31.428421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:42:31.430228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:42:31.500198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:42:31.501649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:42:31.510334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1299/2579] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:42:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:42:35.318053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:42:35.319739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:42:35.423256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80968ced-c405-422b-b263-5f1137b94f52 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:42:35.428181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:42:35.428774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:42:35.429047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:42:35.429354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:42:35.431130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:42:35.443399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:42:35.445004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:42:35.455647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1281/2580] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:42:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:42:41.506413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=46 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:42:45.451444 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=48 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:31.544691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:31.563403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df846d35-c730-4c2f-bfd9-b2128446e6ce service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:31.567828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:31.568421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:31.568697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:31.568968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:31.570835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:31.653461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:31.654881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:31.663672 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1300/2581] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:44:31.729824 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:31.749397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254014a2-83a4-4f0b-b9e0-463ce4025ea1 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:31.753065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:31.753703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:31.754096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:31.754359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:31.762749 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:31.764190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:31.773355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1282/2582] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:44:32.106494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:32.116175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335e97f-bc71-4c99-85cd-4fe15bce4281 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:32.119785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:32.120376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:32.120655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:32.120928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:32.122681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:32.184869 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:32.186382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:32.195819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1301/2583] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:44:35.065491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.080486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5ea35-6840-4d86-b41b-b816adda2910 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.086512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.087419 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.087866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.088298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.090289 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:35.104848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.107230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.123802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1283/2584] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:44:35.138307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.139970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:35.162707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647184a7-bc30-4123-94d3-2a24494e9798 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.171817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.172477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.172757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.173033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] PATH_INFO: `/v3/users/2096dd4d4b6f49fc8572a4bcd9689248` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.174295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2096dd4d4b6f49fc8572a4bcd9689248)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.180233 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.237319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2096dd4d4b6f49fc8572a4bcd9689248'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.237907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2096dd4d4b6f49fc8572a4bcd9689248'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.242849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:35.245554 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2096dd4d4b6f49fc8572a4bcd9689248'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.246301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2096dd4d4b6f49fc8572a4bcd9689248'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.258033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d38fd23-4089-4cec-b665-154dd29702ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2096dd4d4b6f49fc8572a4bcd9689248'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.278068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1302/2585] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:44:35 2022] DELETE /identity/v3/users/2096dd4d4b6f49fc8572a4bcd9689248 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:44:35.470734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.483414 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad751405-4ddf-4065-b3c9-d253b0779c1c service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.553894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.554534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.554816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.555092 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] PATH_INFO: `/v3/projects/e2774daeaf2945eea5b6f33ad4ea2bdf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.556223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2774daeaf2945eea5b6f33ad4ea2bdf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.558177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.605916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2774daeaf2945eea5b6f33ad4ea2bdf', 'request_id': 'req-0d5ffa10-473f-42e7-953a-886473458d09'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.612250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2774daeaf2945eea5b6f33ad4ea2bdf', 'request_id': 'req-0d5ffa10-473f-42e7-953a-886473458d09'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.618441 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2774daeaf2945eea5b6f33ad4ea2bdf', 'request_id': 'req-0d5ffa10-473f-42e7-953a-886473458d09'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.631169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] The token cache is being invalidate because project e2774daeaf2945eea5b6f33ad4ea2bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:44:35.631464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.632118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0d5ffa10-473f-42e7-953a-886473458d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:44:35.633646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1284/2586] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:44:35 2022] DELETE /identity/v3/projects/e2774daeaf2945eea5b6f33ad4ea2bdf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:44:35.639552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.639909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.640259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.660087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:44:35.661360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:35.721927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1303/2587] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:44:35.729886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.730171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.730450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.750628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:44:35.752005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:35.763524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1285/2588] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:44:35.767550 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.777463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84970e3-ccd2-490d-8c87-0b9da7da2b4c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.781963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.782547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.782823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.783089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.783711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:44:35.784006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.785669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.796741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1304/2589] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:44:35.801372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.810406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567c3a3-39f3-4205-9082-af775971d8be None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.815329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.815921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.816192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.817107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.817763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.819636 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.846469 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1286/2590] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:44:35 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:44:35.850716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.860620 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6cc971-7c0d-4f06-a619-ed18393ea74d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.920067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.920671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.920950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.921282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.921908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.923822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.944533 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1305/2591] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:44:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:44:35.948771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.959195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901a11-e9f3-46c5-89b6-e04a24b92113 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.963798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.964390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.964667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.964938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.965609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:35.967193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:35.977434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1287/2592] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:44:35.981326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:35.992082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e5176-46ba-4d5d-8f85-2c44171f719f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:35.996720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:35.997350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:35.997629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:35.997901 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:35.998559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:36.000357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af64fbb-9d78-451b-9bf2-97b516302771 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:36.010963 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1306/2593] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:44:36.014700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:36.024920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e91db-45a3-4b82-8d97-c5814ac3f8ab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:36.029575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:36.030211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:36.030488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:36.030761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] PATH_INFO: `/v3/projects/f6b093eeb2194273b2fa8b00b1955c67/users/43e020686f77463b9315eb26ac1be66e/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:36.042409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:36.047131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b093eeb2194273b2fa8b00b1955c67, user_id=43e020686f77463b9315eb26ac1be66e, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:36.050324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:36.063975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1288/2594] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:44:36 2022] PUT /identity/v3/projects/f6b093eeb2194273b2fa8b00b1955c67/users/43e020686f77463b9315eb26ac1be66e/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:44:36.129565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:36.129849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:36.130125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:36.158248 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] MFA Rules not processed for user `43e020686f77463b9315eb26ac1be66e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:44:36.159483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:36.183642 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:36.224317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1307/2595] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:44:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:44:36.236265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:36.238745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:36.337033 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efff1b-df9d-4e4b-97e3-a8c6efa0bb46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:36.342517 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:36.343112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:36.343392 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:36.343666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:36.351826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:36.353438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:36.362007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1289/2596] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:44:46.220945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=16 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:46.358124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=50,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:44:58.820847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:44:58.822260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:58.877918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47604a49-5d07-4a52-b8ea-5de70868267c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:44:58.882089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:44:58.882682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:44:58.882959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:44:58.883230 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:44:58.884922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:44:58.898044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:44:58.899452 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:44:58.910113 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1308/2597] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:44:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:45:08.905375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=26 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:34.880345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:34.881862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:34.938061 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d4501-5df8-4188-9850-52ca19c69fc7 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:34.941658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:34.942257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:34.942535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:34.942806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:34.944537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:34.953738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:34.955170 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116cd8d-eb64-4c14-8a3f-62098146a76a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:34.963516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1290/2598] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:45:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:45:36.298380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:36.359603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3111d-5a4b-4562-a7df-684bc47ce70f service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:36.364084 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:36.364703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:36.364979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:36.365278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] PATH_INFO: `/v3/users/43e020686f77463b9315eb26ac1be66e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:36.366405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e020686f77463b9315eb26ac1be66e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:36.368321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:36.414134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e020686f77463b9315eb26ac1be66e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.414787 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e020686f77463b9315eb26ac1be66e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.421542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e020686f77463b9315eb26ac1be66e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.422188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e020686f77463b9315eb26ac1be66e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.428976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e020686f77463b9315eb26ac1be66e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.445686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1309/2599] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:45:36 2022] DELETE /identity/v3/users/43e020686f77463b9315eb26ac1be66e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:45:36.455168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:36.455715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:36.455715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:36.476963 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:45:36.478201 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:36.534501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1291/2600] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:45:36.545158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:36.546537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:36.616486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d65a13-ca3b-43ee-a435-5cb042e8479d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:36.620122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:36.620724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:36.621044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:36.621307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:36.630448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:36.632374 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943c50f-5b70-43c8-857a-21570e058369 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:36.641691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1310/2601] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:45:36.909147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:36.910702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:36.923023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7c457-7858-4c87-ad2a-56fc15fe0ca8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:36.928417 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:36.929252 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:36.929529 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:36.929795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] PATH_INFO: `/v3/projects/f6b093eeb2194273b2fa8b00b1955c67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:36.930921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b093eeb2194273b2fa8b00b1955c67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:36.936126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:36.983186 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b093eeb2194273b2fa8b00b1955c67', 'request_id': 'req-0b3adb6f-9a41-4039-ab44-24373d999353'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.989233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b093eeb2194273b2fa8b00b1955c67', 'request_id': 'req-0b3adb6f-9a41-4039-ab44-24373d999353'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:36.994608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b093eeb2194273b2fa8b00b1955c67', 'request_id': 'req-0b3adb6f-9a41-4039-ab44-24373d999353'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:37.006251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] The token cache is being invalidate because project f6b093eeb2194273b2fa8b00b1955c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:45:37.006536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:37.007149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0b3adb6f-9a41-4039-ab44-24373d999353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:45:37.008620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1292/2602] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:45:36 2022] DELETE /identity/v3/projects/f6b093eeb2194273b2fa8b00b1955c67 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:45:37.013586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.013875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.014154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.034092 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:45:37.035296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:37.044655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:37.095999 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1311/2603] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:45:37.104145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.104427 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.104701 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.123855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:45:37.125055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:37.133307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1293/2604] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:45:37.137252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.147809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6faa2-3794-4e7c-9214-d03bc0bd78f4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.152616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.153182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.153466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.153741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.154407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:45:37.154712 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.156334 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.167826 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1312/2605] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:45:37.172122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.181295 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7894-d3dc-4a0b-9288-e746d0aa0cdf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.186064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.186719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.187000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.187271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.187901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.189833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.212210 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:37.221374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1294/2606] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:45:37.224773 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.243561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32577c-bf22-462e-980d-e8e8bf5d29a3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.301059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.301753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.301958 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.302225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.303272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.305466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.327625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1313/2607] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:45:37.331627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.343673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e700101-ee60-4ba7-80cb-f9c8052bdfd3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.348485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.349090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.349395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.349668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.350335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.351925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.361866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1295/2608] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:45:37.365923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.376296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26d90c-a93b-4e5a-bb5a-d7a75f7d4f8e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.380909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.382044 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.382328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.382604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.383243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.384829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7deaa2-575e-4ec4-b61a-580d8d1d9e63 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.396222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1314/2609] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:45:37.401250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.411251 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f935e63-7396-49ae-83f4-14c3cd805b61 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.415973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.416654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.417153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.417292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] PATH_INFO: `/v3/projects/19dd55ee6f9b43be92ead627b360b138/users/5c1d7a22661f418388826daebe39a4d1/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.432354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=19dd55ee6f9b43be92ead627b360b138, user_id=5c1d7a22661f418388826daebe39a4d1, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.435129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.448471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1296/2610] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:45:37 2022] PUT /identity/v3/projects/19dd55ee6f9b43be92ead627b360b138/users/5c1d7a22661f418388826daebe39a4d1/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:45:37.458301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.458664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.458938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.486859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] MFA Rules not processed for user `5c1d7a22661f418388826daebe39a4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:45:37.488158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:37.552714 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1315/2611] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:45:37.557675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.617471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a000-c4b4-4057-adf7-e2888ff471ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.622019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.622617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.622897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.623172 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.623793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.625716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.652012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1297/2612] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/projects => generated 403 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:45:37.656147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.666647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b715d-f4e6-4029-ae20-90584f5ed7e5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.671140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:37.722715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.723313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.723591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.723861 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.724492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.726486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.746655 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1316/2613] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/users => generated 308 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:45:37.750546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.761513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b39b862-b21f-44d5-b536-98fd43671a55 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.766156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.766759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.767501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.767501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.767919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.769553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.780375 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1298/2614] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:45:37.784468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.794389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f852933-e674-4307-ba9d-fe2cc10d5ab0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.798986 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.799629 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.799896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.800169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] PATH_INFO: `/v3/projects/10c651cf5f334e70ba394cdcd6e53da2/users/cf49d33b891f4f5eb186f34562b22e06/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.814894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c651cf5f334e70ba394cdcd6e53da2, user_id=cf49d33b891f4f5eb186f34562b22e06, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.817690 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.831375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1317/2615] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:45:37 2022] PUT /identity/v3/projects/10c651cf5f334e70ba394cdcd6e53da2/users/cf49d33b891f4f5eb186f34562b22e06/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:45:37.835260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.847949 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680a39-b8a1-4fd1-8c74-c28794a00047 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.903731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.904331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.904609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.904883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.905552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.907187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfa429-f54b-4342-9295-ab17e826bd54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.917135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1299/2616] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:45:37.921328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:45:37.932095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ee4c4-df0b-4b5d-88c0-7a78401471ea admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:45:37.936709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:45:37.937390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.937668 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.937943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] PATH_INFO: `/v3/projects/10c651cf5f334e70ba394cdcd6e53da2/users/cf49d33b891f4f5eb186f34562b22e06/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.939855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c651cf5f334e70ba394cdcd6e53da2, user_id=cf49d33b891f4f5eb186f34562b22e06, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:45:37.942538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:45:37.952685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1318/2617] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:45:37 2022] PUT /identity/v3/projects/10c651cf5f334e70ba394cdcd6e53da2/users/cf49d33b891f4f5eb186f34562b22e06/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:45:37.963796 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:45:37.964085 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:45:37.964362 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:45:37.993088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] MFA Rules not processed for user `cf49d33b891f4f5eb186f34562b22e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:45:37.994319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:45:38.013476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:38.062027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1300/2618] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:45:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:45:47.948996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=40,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:45:48.058956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=19 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:32.012307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:32.014235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:32.074225 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065446b0-6a1a-491a-bff5-888a0ed617b0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:32.077813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:32.078415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:32.078693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:32.078964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:32.080741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:32.145959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:32.148204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:32.165859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1319/2619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:46:32.310654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:32.320793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9e708-fafc-452e-bce9-872f6cdbfb9c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:32.324368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:32.324909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:32.325228 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:32.325504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:32.334606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:32.336149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:32.345484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1301/2620] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:46:32.661267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:32.663288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:32.719573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217945a4-988a-48d6-8c2e-f1d507b4d11a service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:32.723183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:32.723771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:32.724043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:32.724309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:32.726524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:32.774928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:32.789504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:32.790924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:32.800179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1320/2621] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:46:35.751967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:35.754216 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:35.840771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8090-a454-4391-825e-5eb01d94e3e8 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:35.846191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:35.847070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:35.847524 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:35.847978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:35.850236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:35.862518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:35.864112 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:35.874921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1302/2622] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:46:38.980151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:38.980151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:38.993716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50503630-0751-4ca9-b42f-cfe235b36698 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:38.998477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:38.999104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:38.999380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:38.999650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] PATH_INFO: `/v3/users/d3a4f1b15f0442578e65b810edfb4923` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:39.000749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a4f1b15f0442578e65b810edfb4923)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:39.002786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:39.051338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a4f1b15f0442578e65b810edfb4923'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.052021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a4f1b15f0442578e65b810edfb4923'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.059563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a4f1b15f0442578e65b810edfb4923'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.060804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a4f1b15f0442578e65b810edfb4923'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.068218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a4f1b15f0442578e65b810edfb4923'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.086504 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1321/2623] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:46:38 2022] DELETE /identity/v3/users/d3a4f1b15f0442578e65b810edfb4923 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:39.256531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:39.273487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7fda-a458-4273-b17a-f961191ab43a service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:39.330263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:39.330885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:39.331163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:39.331436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] PATH_INFO: `/v3/projects/bbc5effaf211410eb0b8872c2be553da` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:39.332540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc5effaf211410eb0b8872c2be553da)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:39.334559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:39.383298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc5effaf211410eb0b8872c2be553da', 'request_id': 'req-4ae51ee8-4c70-4a45-b9b1-229a61523552'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.390019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc5effaf211410eb0b8872c2be553da', 'request_id': 'req-4ae51ee8-4c70-4a45-b9b1-229a61523552'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.395887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc5effaf211410eb0b8872c2be553da', 'request_id': 'req-4ae51ee8-4c70-4a45-b9b1-229a61523552'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.409233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] The token cache is being invalidate because project bbc5effaf211410eb0b8872c2be553da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:46:39.409526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.410180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.412048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1303/2624] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:46:39 2022] DELETE /identity/v3/projects/bbc5effaf211410eb0b8872c2be553da => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:39.415772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:39.417354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:39.430301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652e84c-baa7-45e1-a95e-2c856d5b40cf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:39.488210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:39.488856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:39.489249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:39.489577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] PATH_INFO: `/v3/users/b09bb0c8090c4d4088da847e972c2aa2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:39.490723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09bb0c8090c4d4088da847e972c2aa2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:39.492794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:39.538338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09bb0c8090c4d4088da847e972c2aa2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.538984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09bb0c8090c4d4088da847e972c2aa2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.545770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09bb0c8090c4d4088da847e972c2aa2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.546391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09bb0c8090c4d4088da847e972c2aa2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.550002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:39.554345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24ef97ad-d7b5-4eca-b250-45f8f990e6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09bb0c8090c4d4088da847e972c2aa2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.580560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1322/2625] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:46:39 2022] DELETE /identity/v3/users/b09bb0c8090c4d4088da847e972c2aa2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:39.751780 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:39.753355 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:39.766784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51ee8-4c70-4a45-b9b1-229a61523552 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:39.770576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:39.826904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:39.827527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:39.827803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:39.828075 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] PATH_INFO: `/v3/projects/09bdfceb1dd84035a3cf16bc3c261292` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:39.829208 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09bdfceb1dd84035a3cf16bc3c261292)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:39.831101 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:39.874833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09bdfceb1dd84035a3cf16bc3c261292', 'request_id': 'req-8d1637db-f17d-492f-9e58-018626acd69c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.882634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09bdfceb1dd84035a3cf16bc3c261292', 'request_id': 'req-8d1637db-f17d-492f-9e58-018626acd69c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.888813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09bdfceb1dd84035a3cf16bc3c261292', 'request_id': 'req-8d1637db-f17d-492f-9e58-018626acd69c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.901478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] The token cache is being invalidate because project 09bdfceb1dd84035a3cf16bc3c261292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:46:39.901766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.902420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8d1637db-f17d-492f-9e58-018626acd69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:46:39.903923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1304/2626] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:46:39 2022] DELETE /identity/v3/projects/09bdfceb1dd84035a3cf16bc3c261292 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:39.908836 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:39.909144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:39.909426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:39.929638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:46:39.930852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:39.988474 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1323/2627] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:46:39.996671 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:39.996955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:39.997258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.016975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:46:40.018215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:40.026217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1305/2628] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:46:40.030271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.039931 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fca6c3-f212-4f90-8a60-636e1d224dd2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.044506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.045100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.045409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.045685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.046324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:46:40.046627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.048257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.060060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1324/2629] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:46:40.064052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.075016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009053da-6cba-4c3b-86a8-610acf2a2828 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.080002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.080616 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.080895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.081196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.081847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.083836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.119485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1306/2630] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:46:40 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:46:40.123838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.133972 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814dc5-fd44-414a-af33-db4f9c541061 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.189774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.190372 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.190651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.190922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.191538 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.193560 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.213757 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1325/2631] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:46:40 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:46:40.218099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.228551 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4eb3-5609-4449-b410-3ce37682bb2f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.233048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.233665 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.233939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.234207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.234837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.236415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.247322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1307/2632] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:46:40.251108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.260896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31a6b-6138-4aa5-8b7e-9436dbf4e8ee admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.265507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.266095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.266373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.266644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.267272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.268860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce240e4d-a2cf-44f5-8db1-8f1881bf8d06 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.279835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1326/2633] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:46:40.283022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.294090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196f1e-0ca7-44b0-9745-b7b4d33528cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.298730 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.299376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.299651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.299921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] PATH_INFO: `/v3/projects/df5069d6a3a34a08bb00105a91251cf2/users/93b2cf587dc44209a186fc453ac5bdfc/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.314857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5069d6a3a34a08bb00105a91251cf2, user_id=93b2cf587dc44209a186fc453ac5bdfc, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.317537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.331373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1308/2634] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:46:40 2022] PUT /identity/v3/projects/df5069d6a3a34a08bb00105a91251cf2/users/93b2cf587dc44209a186fc453ac5bdfc/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:40.336370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45312-9b01-4075-ba3e-a3c7c9f2da7d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.336660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45312-9b01-4075-ba3e-a3c7c9f2da7d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.336965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45312-9b01-4075-ba3e-a3c7c9f2da7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.357744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0ac45312-9b01-4075-ba3e-a3c7c9f2da7d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:46:40.358953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0ac45312-9b01-4075-ba3e-a3c7c9f2da7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:40.401199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:40.422180 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1327/2635] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:46:40.431915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:40.433692 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:40.469384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:40.492651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b476c-8f10-4d49-8dd8-5a37f6da8209 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:40.496257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:40.496861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:40.497163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:40.497486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:40.506443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:40.507862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:40.517148 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1309/2636] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:46:44.365477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.366078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.366078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.400928 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] MFA Rules not processed for user `93b2cf587dc44209a186fc453ac5bdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:46:44.402219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:44.471671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1328/2637] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:46:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:46:44.476302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.486187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e7b42-38ab-48fa-b011-cf6ba7b4c423 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.491519 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.492235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.492739 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.493013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.493685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.495605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.528681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1310/2638] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:46:44 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:46:44.530036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.542957 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66894323-fdf4-4092-9c7c-074310acbac4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.602450 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.603060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.603340 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.603644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.604271 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.606239 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.629739 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1329/2639] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:46:44 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:46:44.634274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.644782 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26625b99-931f-4120-b4d9-030238bcf6db admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.650303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.650923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.651199 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.651513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.652131 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.653754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.664728 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1311/2640] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:46:44.668816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.679188 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f997f-53ed-4ab3-9f92-8417cba72fd7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.683837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.684481 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.684761 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.685033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] PATH_INFO: `/v3/projects/9791973d73d74a3095f3e6346850f347/users/f3324c5422254ead8a05136fce2856de/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.700228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791973d73d74a3095f3e6346850f347, user_id=f3324c5422254ead8a05136fce2856de, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.702970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.717045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1330/2641] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:46:44 2022] PUT /identity/v3/projects/9791973d73d74a3095f3e6346850f347/users/f3324c5422254ead8a05136fce2856de/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:46:44.721010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.730940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0682668-8e7b-4daa-b253-71d0ae31b6a1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.786207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.787346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.787625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.787890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.788512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.790119 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840ece7-91a3-4788-91ac-0f1ccc467fbe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.799562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1312/2642] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:46:44.804221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:44.814346 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddba34-bed4-46f1-8487-833d82e7385e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:44.818889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:44.819528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:44.819800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:44.820067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] PATH_INFO: `/v3/projects/9791973d73d74a3095f3e6346850f347/users/f3324c5422254ead8a05136fce2856de/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:44.821985 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9791973d73d74a3095f3e6346850f347, user_id=f3324c5422254ead8a05136fce2856de, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:44.824595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:44.834536 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1331/2643] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:46:44 2022] PUT /identity/v3/projects/9791973d73d74a3095f3e6346850f347/users/f3324c5422254ead8a05136fce2856de/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:46:49.258994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea98-c682-4292-b548-150043c1abfd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:49.259536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea98-c682-4292-b548-150043c1abfd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:49.259677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea98-c682-4292-b548-150043c1abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:49.291286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a15aea98-c682-4292-b548-150043c1abfd None None] MFA Rules not processed for user `f3324c5422254ead8a05136fce2856de`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:46:49.292565 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a15aea98-c682-4292-b548-150043c1abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:49.368418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1313/2644] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:46:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:46:49.384617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:46:49.386604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:46:49.431570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:49.483719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b407025-e214-4e58-ab99-ea52645bb986 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:46:49.489421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:46:49.490577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:46:49.490849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:46:49.491127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:46:49.558268 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:46:49.560013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:46:49.570573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1332/2645] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:46:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:46:59.364847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:46:59.565850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=44 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:47:39.211445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aea98-c682-4292-b548-150043c1abfd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:47:39.212841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a15aea98-c682-4292-b548-150043c1abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:47:39.282191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aea98-c682-4292-b548-150043c1abfd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:47:39.287133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:47:39.287769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:47:39.288050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:47:39.288328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:47:39.290069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:47:39.301763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:47:39.303330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:47:39.313632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1314/2646] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:47:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:47:49.308842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:48:32.765867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:48:32.767322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:48:32.833369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d71d2-f542-483f-b277-1243a2c44e20 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:48:32.837097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:48:32.837710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:48:32.837984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:48:32.838251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:48:32.839980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:48:32.906453 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:48:32.907867 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e6c26-38dd-4d0a-8ba5-5e21aa5beebd service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:48:32.925791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1333/2647] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:48:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:48:42.923103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=60 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:00.904205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:00.905690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:01.023839 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6aad6-1df2-4b85-b8a6-1e13246cf5f0 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:01.028409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:01.029037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:01.029343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:01.029618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] PATH_INFO: `/v3/users/5c1d7a22661f418388826daebe39a4d1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:01.030746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1d7a22661f418388826daebe39a4d1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:01.032657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:01.078773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1d7a22661f418388826daebe39a4d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.086096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1d7a22661f418388826daebe39a4d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.093283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1d7a22661f418388826daebe39a4d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.093901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1d7a22661f418388826daebe39a4d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.106214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1d7a22661f418388826daebe39a4d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.111984 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1315/2648] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:00 2022] DELETE /identity/v3/users/5c1d7a22661f418388826daebe39a4d1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:01.116492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:01.116989 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:01.117132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:01.144762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:01.146010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:01.205041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1334/2649] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:01.215258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:01.216673 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:01.273807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f2433-30c3-411b-b230-afda5994332e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:01.277868 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:01.278476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:01.278752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:01.279024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:01.284858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:01.289934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:01.291504 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:01.302989 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1316/2650] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:01.568910 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:01.570424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:01.582510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ee70e-2e78-4b9c-a24d-f169df80803e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:01.589942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:01.590601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:01.590876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:01.591149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] PATH_INFO: `/v3/projects/19dd55ee6f9b43be92ead627b360b138` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:01.592326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19dd55ee6f9b43be92ead627b360b138)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:01.594369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:01.641493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19dd55ee6f9b43be92ead627b360b138', 'request_id': 'req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.649748 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19dd55ee6f9b43be92ead627b360b138', 'request_id': 'req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.655679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19dd55ee6f9b43be92ead627b360b138', 'request_id': 'req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.667017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] The token cache is being invalidate because project 19dd55ee6f9b43be92ead627b360b138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:01.667304 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.667938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.669455 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1335/2651] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:01 2022] DELETE /identity/v3/projects/19dd55ee6f9b43be92ead627b360b138 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:01.673461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:01.683570 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99d44-0d8b-4825-b11d-8b14e92c166b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:01.740071 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:01.740699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:01.740978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:01.741292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] PATH_INFO: `/v3/users/cf49d33b891f4f5eb186f34562b22e06` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:01.743146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf49d33b891f4f5eb186f34562b22e06)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:01.745254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:01.797964 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf49d33b891f4f5eb186f34562b22e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.804910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf49d33b891f4f5eb186f34562b22e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.813961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf49d33b891f4f5eb186f34562b22e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.814680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf49d33b891f4f5eb186f34562b22e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.827360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a5fe6ed1-e0d1-4b53-a9a8-04a547571607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf49d33b891f4f5eb186f34562b22e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:01.833274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1317/2652] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:01 2022] DELETE /identity/v3/users/cf49d33b891f4f5eb186f34562b22e06 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:49:02.102745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.104783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.120137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93323cb3-b8a6-4045-a3ea-fb1f439edf2f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.186096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.186726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.187004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.187278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] PATH_INFO: `/v3/projects/10c651cf5f334e70ba394cdcd6e53da2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.188377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c651cf5f334e70ba394cdcd6e53da2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.190322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.211581 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:02.236371 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c651cf5f334e70ba394cdcd6e53da2', 'request_id': 'req-2c41188e-19b8-45f2-9f96-65d90660f8c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:02.242997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c651cf5f334e70ba394cdcd6e53da2', 'request_id': 'req-2c41188e-19b8-45f2-9f96-65d90660f8c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:02.248829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c651cf5f334e70ba394cdcd6e53da2', 'request_id': 'req-2c41188e-19b8-45f2-9f96-65d90660f8c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:02.260515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] The token cache is being invalidate because project 10c651cf5f334e70ba394cdcd6e53da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:02.260938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:02.261635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2c41188e-19b8-45f2-9f96-65d90660f8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:02.263129 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1336/2653] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:02 2022] DELETE /identity/v3/projects/10c651cf5f334e70ba394cdcd6e53da2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:02.267953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.268451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.268584 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.319054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:02.320939 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.366871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:02.385141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1318/2654] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:49:02.392652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.392935 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.393237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.415671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:02.417779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.426389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1337/2655] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:02.430651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.440348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b63587-3a7f-4d5e-bceb-add1bd4d5e20 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.445726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.446979 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.447272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.447542 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.448187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:02.448489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.450109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.462628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1319/2656] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:02.466730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.476246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfaf6e-c2b9-48ea-8726-f3b4e79e127f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.481615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.482247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.482524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.482793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.483438 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.485517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.521947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1338/2657] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:02.526813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.536962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a025f4a-1f45-404f-b376-5055b4ef60a3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.596182 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.596811 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.597086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.597382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.598007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.599917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.622260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1320/2658] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:02.626337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.637427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d72a43-739c-4180-95c8-92881c9c6feb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.642891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.643772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.644119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.644470 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.645207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.648364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.662473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1339/2659] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:02.666368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.680029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1227-15dd-42bc-a72e-91406774a6de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.684718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.685349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.685626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.686508 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.687160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.688863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.701497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1321/2660] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:02.705654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.722106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.725234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.727036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c96d1-6e05-413a-85c8-2ed2cbd99897 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.732380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.733150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.733343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.733607 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] PATH_INFO: `/v3/projects/29a0f80e80994af2ac94ff159298b595/users/6646bda4e01c4f0c91255d0086cac7f7/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.751181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a0f80e80994af2ac94ff159298b595, user_id=6646bda4e01c4f0c91255d0086cac7f7, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.754149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86687944-ebd2-4e1c-9f16-188e1fb5ca71 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.773520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1340/2661] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:02 2022] PUT /identity/v3/projects/29a0f80e80994af2ac94ff159298b595/users/6646bda4e01c4f0c91255d0086cac7f7/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:02.783492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.783897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.784219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.809521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43869430-1e21-459b-96b5-54c48cdf047d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.816345 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] MFA Rules not processed for user `6646bda4e01c4f0c91255d0086cac7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:02.817602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.859019 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:02.878005 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.878618 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.878895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.879167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.881046 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:02.885014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1341/2662] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:49:02.889661 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:02.893588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.895188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.906200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1322/2663] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:02.918917 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:02.956388 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66adb1d7-2c09-4acc-9238-1b3eee5e078d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:02.961638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:02.962240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:02.962520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:02.962790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:02.963418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:02.965347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:02.993923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1342/2664] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:49:02.997891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.010527 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3866cd4-eac6-40da-9ec6-9ec5c0489bd0 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.071177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.071797 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.072076 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.072351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.072981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.074967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.098714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1323/2665] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:02 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:03.103494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.114381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645fa04-d600-4eb1-ba9a-4129767a81b2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.119769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.120444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.120988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.121356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.122003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.123859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.140387 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1343/2666] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:03.144238 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.154627 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7267f-6260-459a-9696-ed8261d9685e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.159651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.160292 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.160576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.160849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] PATH_INFO: `/v3/projects/0e30d43d174347fdb08375d661d4c9ee/users/4335a0c58dec4301aad4c519ada88f42/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.175079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e30d43d174347fdb08375d661d4c9ee, user_id=4335a0c58dec4301aad4c519ada88f42, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.177756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.190829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1324/2667] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:03 2022] PUT /identity/v3/projects/0e30d43d174347fdb08375d661d4c9ee/users/4335a0c58dec4301aad4c519ada88f42/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:03.195611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.205480 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122750-6e38-4c78-a967-3043d62cd15c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.261621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.262143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.262418 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.262691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.263310 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.264874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2f7df-8508-4a38-a4bb-9fc30ebb5877 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.274580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1344/2668] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:03.278870 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.290354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329bc9e-9d6a-4331-a780-21de8713cd77 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.295434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.296126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.296404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.296679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] PATH_INFO: `/v3/projects/0e30d43d174347fdb08375d661d4c9ee/users/4335a0c58dec4301aad4c519ada88f42/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.298654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e30d43d174347fdb08375d661d4c9ee, user_id=4335a0c58dec4301aad4c519ada88f42, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.301311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed5a66-1d82-460d-adb9-874ee380933e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.312767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1325/2669] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:03 2022] PUT /identity/v3/projects/0e30d43d174347fdb08375d661d4c9ee/users/4335a0c58dec4301aad4c519ada88f42/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:03.324571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.324886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.325196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.354140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] MFA Rules not processed for user `4335a0c58dec4301aad4c519ada88f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:03.355421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:03.422053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1345/2670] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:03.648744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.649028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.649325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.688348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] MFA Rules not processed for user `c6a74f33e6e941ed8c3f7850bdf29bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:03.689589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:03.758575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1326/2671] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:03.862862 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:03.864320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:03.891650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:03.939916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60cfaf-adeb-4bdf-890a-78f8992fe0ab None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:03.943899 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:03.944510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:03.944822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:03.945272 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:03.955440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:03.956865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:03.967478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1346/2672] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:04.046808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:04.056488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2c464-b417-45a7-a647-716cc3c06a87 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:04.056730 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:04.059144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:04.060151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:04.060756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:04.061028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:04.061332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:04.076372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:04.121819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef23cf-6d65-421e-8155-ef0c6ff50e4c service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:04.127713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:04.129025 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:04.129341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:04.129490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:04.129615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:04.131286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:04.131797 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:04.146559 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1327/2673] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:04.148037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:04.150333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:04.163570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1347/2674] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:04.526476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:04.536360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed9a7e-8a2c-44cf-89db-e54569d3e7ef service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:04.550247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:04.550858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:04.551141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:04.551415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:04.553299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:04.635146 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:04.636597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:04.658961 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1328/2675] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:05.503998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:05.505424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:05.574403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72d8d8-b380-40d2-8fbf-d443dd7e868f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:05.579270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:05.579858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:05.580133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:05.580407 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:05.582124 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:05.593631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:05.595174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b39f3-6fd1-43a9-9fb3-86f81dbd2b3c service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:05.605389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1348/2676] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:06.616882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:06.631842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31442-d6e0-44a6-b287-c5a6e0d68891 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:06.641280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:06.643404 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:06.644133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:06.644547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:06.646658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:06.662624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:06.664792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:06.676307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1329/2677] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:12.197302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:12.197833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:12.197833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:12.218575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] MFA Rules not processed for user `93b2cf587dc44209a186fc453ac5bdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:12.219794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:12.246353 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:12.255820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6f96-190f-4bb6-a22b-4641849815ac service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:12.259503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:12.260098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:12.260374 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:12.260647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:12.262408 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:12.272132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:12.284598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1349/2678] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:12.327048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:12.328497 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:12.339592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1330/2679] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:49:16.739638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:16.741709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:16.782571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d414e70-1d96-426d-bd98-0ab1b5e243d3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:16.790439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:16.791323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:16.791813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:16.792439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] PATH_INFO: `/v3/users/db4f0928502a43b490de01afc587a846` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:16.794000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4f0928502a43b490de01afc587a846)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:16.796427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:16.880237 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4f0928502a43b490de01afc587a846'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:16.880908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4f0928502a43b490de01afc587a846'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:16.891182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4f0928502a43b490de01afc587a846'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:16.891909 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4f0928502a43b490de01afc587a846'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:16.908264 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d8ac24f7-6b7c-48e4-99b0-65567da5be15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4f0928502a43b490de01afc587a846'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:16.929203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1350/2680] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:16 2022] DELETE /identity/v3/users/db4f0928502a43b490de01afc587a846 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:17.126369 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.136901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1382d4-7d8c-4701-9e27-cfc0b2dc9dea service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.150154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:17.215136 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.215838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.216278 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.216398 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] PATH_INFO: `/v3/projects/c1705474b7954cf1926e9beb2d548587` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.217502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1705474b7954cf1926e9beb2d548587)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.219422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.269742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1705474b7954cf1926e9beb2d548587', 'request_id': 'req-cf7538d0-a87d-4778-8a05-2ea757f1ec69'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:17.277827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1705474b7954cf1926e9beb2d548587', 'request_id': 'req-cf7538d0-a87d-4778-8a05-2ea757f1ec69'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:17.283759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1705474b7954cf1926e9beb2d548587', 'request_id': 'req-cf7538d0-a87d-4778-8a05-2ea757f1ec69'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:17.296727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] The token cache is being invalidate because project c1705474b7954cf1926e9beb2d548587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:17.297028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:17.297707 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf7538d0-a87d-4778-8a05-2ea757f1ec69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:17.299193 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1331/2681] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:17 2022] DELETE /identity/v3/projects/c1705474b7954cf1926e9beb2d548587 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:17.305473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.305760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.306038 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.326688 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:17.327897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:17.397234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1351/2682] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:17.399651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.400004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.400346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.420068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:17.421382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:17.438222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1332/2683] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:17.442490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.453975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0cbb1-b12f-4d16-85e7-8aff7684021b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.460732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.461324 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.461805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.462080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.462733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:17.463033 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.464641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.481270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1352/2684] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:17.483561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.497317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b87e50-22ab-4ff2-8212-b83145ecd509 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.505377 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.506178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.506705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.507795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.508719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.511052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.541886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1333/2685] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:17 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:49:17.546211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.561315 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad597f41-38f2-4881-93e9-aecc6c5b3100 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.623013 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206680-331c-4296-9155-40eb4926376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.623840 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206680-331c-4296-9155-40eb4926376e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.624122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206680-331c-4296-9155-40eb4926376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.624392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46206680-331c-4296-9155-40eb4926376e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.625019 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46206680-331c-4296-9155-40eb4926376e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.626953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46206680-331c-4296-9155-40eb4926376e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.635723 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:17.650566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1353/2686] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:17 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:49:17.654834 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.666282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4b98-19d0-4d55-bde4-bc324d4792d8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.670922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.671521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.671848 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.672309 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.672959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.674645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.685933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1334/2687] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:17.689981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206680-331c-4296-9155-40eb4926376e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.700367 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46206680-331c-4296-9155-40eb4926376e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.705046 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.705691 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.705971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.706241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.706894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.708534 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cad26-6b1a-4963-9a55-df3729207c74 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.718754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1354/2688] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:17.722970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.734498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae4e5d-f907-4ce9-8d76-fe9d768d27ab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:17.740002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:17.740642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.740926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.742595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] PATH_INFO: `/v3/projects/6a0433dc28924fc891b5e114c521bb57/users/4496d5005b0040e8b70020740e9e3401/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.775433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0433dc28924fc891b5e114c521bb57, user_id=4496d5005b0040e8b70020740e9e3401, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:17.779326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:17.813793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1335/2689] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:17 2022] PUT /identity/v3/projects/6a0433dc28924fc891b5e114c521bb57/users/4496d5005b0040e8b70020740e9e3401/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:17.818640 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:17.819606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:17.820657 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:17.844817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:17.847348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:17.959189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1355/2690] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:49:17.966696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:17.968612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:18.075536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:18.106399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d5a8b-34fa-4edd-a452-094c7bc64cca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:18.118543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:18.119279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:18.119625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:18.119973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:18.131946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:18.135235 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:18.149591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1336/2691] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:18.331660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:18.333802 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:18.482514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138cc68-3328-4a06-a9c8-0b011bb880f5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:18.502403 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:18.503829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:18.504805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:18.505351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:18.508645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:18.531117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:18.533741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfa4de-bb41-461a-acb5-6c0b077e4b79 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:18.561919 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1356/2692] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:19.954276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:19.956543 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:20.014145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351be1ab-6d35-4e9e-a971-1fd4679cd8fe service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:20.020785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:20.021862 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:20.022294 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:20.023474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] PATH_INFO: `/v3/users/6646bda4e01c4f0c91255d0086cac7f7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:20.030006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6646bda4e01c4f0c91255d0086cac7f7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:20.031999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:20.112900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6646bda4e01c4f0c91255d0086cac7f7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.121260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6646bda4e01c4f0c91255d0086cac7f7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.132573 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6646bda4e01c4f0c91255d0086cac7f7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.133483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6646bda4e01c4f0c91255d0086cac7f7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.149104 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6646bda4e01c4f0c91255d0086cac7f7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.156177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1337/2693] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:19 2022] DELETE /identity/v3/users/6646bda4e01c4f0c91255d0086cac7f7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:20.160580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:20.161163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:20.161163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:20.193815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:20.195674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:20.259086 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1357/2694] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:20.274276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:20.275871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:20.341466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec9ec-d947-4188-bb4e-3eb6981540c8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:20.345336 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:20.346020 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:20.346358 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:20.346635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:20.355936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:20.357391 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:20.369505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1338/2695] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:20.780376 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:20.782286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:20.791654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:20.796146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989834d7-7b08-41cf-9788-d33915b9c530 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:20.803259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:20.804460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:20.804925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:20.805386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] PATH_INFO: `/v3/projects/29a0f80e80994af2ac94ff159298b595` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:20.806726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a0f80e80994af2ac94ff159298b595)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:20.808783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:20.869769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a0f80e80994af2ac94ff159298b595', 'request_id': 'req-2a87c571-c36a-4e56-8b50-452498785a63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.881612 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a0f80e80994af2ac94ff159298b595', 'request_id': 'req-2a87c571-c36a-4e56-8b50-452498785a63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.888882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a0f80e80994af2ac94ff159298b595', 'request_id': 'req-2a87c571-c36a-4e56-8b50-452498785a63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.901648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] The token cache is being invalidate because project 29a0f80e80994af2ac94ff159298b595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:20.901944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.902616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:20.904460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1358/2696] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:20 2022] DELETE /identity/v3/projects/29a0f80e80994af2ac94ff159298b595 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:20.909806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:20.921515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ee834-b868-4e50-b13c-7e0b5ae988e7 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:20.986286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:20.987011 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:20.987368 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:20.987710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] PATH_INFO: `/v3/users/4335a0c58dec4301aad4c519ada88f42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:20.988931 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4335a0c58dec4301aad4c519ada88f42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:20.992149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:20.996521 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:21.056334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4335a0c58dec4301aad4c519ada88f42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:21.069426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4335a0c58dec4301aad4c519ada88f42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:21.079881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4335a0c58dec4301aad4c519ada88f42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:21.080808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4335a0c58dec4301aad4c519ada88f42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:21.099915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b93bb8e1-6b72-4bfa-bc2b-fe4dd5d94ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4335a0c58dec4301aad4c519ada88f42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:21.109105 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1339/2697] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:20 2022] DELETE /identity/v3/users/4335a0c58dec4301aad4c519ada88f42 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:22.065539 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.067255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:22.088768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87c571-c36a-4e56-8b50-452498785a63 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.186167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.186895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.187242 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.187570 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] PATH_INFO: `/v3/projects/0e30d43d174347fdb08375d661d4c9ee` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.189397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e30d43d174347fdb08375d661d4c9ee)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.193358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.247366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e30d43d174347fdb08375d661d4c9ee', 'request_id': 'req-eeaf1319-3619-4aaa-abc3-8265726cdee1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:22.256740 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e30d43d174347fdb08375d661d4c9ee', 'request_id': 'req-eeaf1319-3619-4aaa-abc3-8265726cdee1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:22.265253 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e30d43d174347fdb08375d661d4c9ee', 'request_id': 'req-eeaf1319-3619-4aaa-abc3-8265726cdee1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:22.284633 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] The token cache is being invalidate because project 0e30d43d174347fdb08375d661d4c9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:22.285068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:22.286107 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eeaf1319-3619-4aaa-abc3-8265726cdee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:22.294138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1359/2698] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:22 2022] DELETE /identity/v3/projects/0e30d43d174347fdb08375d661d4c9ee => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:22.294351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.294351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.294351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.324578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:22.326342 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:22.414918 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1340/2699] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:22.422158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.422444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.422720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.443760 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:22.445055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:22.457617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1360/2700] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:22.462241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.473400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445f22-80a8-4203-913d-ff389cee70ea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.479042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.480102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.480645 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.481207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.481924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:22.482299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.484171 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.498668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1341/2701] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:22.503291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.513466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddead8b9-d273-4b4f-926f-bf1e2999957c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.518632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.519356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.520157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.520157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.520824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.523716 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.534142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:22.561270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1361/2702] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:22.563630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.577088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466c8ff-07d3-4d47-85a0-3aa6ea879cd9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.645889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.646594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.646941 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.647428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.649043 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.651088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.666065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:22.681154 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1342/2703] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:22.684784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.695721 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6688e8-c5d8-444f-ada7-be76847d86c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.701510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.702149 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.702454 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.702832 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.703701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.705012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.716207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1362/2704] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:22.720200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.731663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c36f28-fa53-47d3-b02f-8735969a6212 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.737330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.738364 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.738630 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.739072 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.740069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.741960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a03aa-9b0f-4d5b-ab3d-f0b4ecb6cd69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.758788 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1343/2705] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:22.760009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:22.776288 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03023-44b6-453c-bfb7-2835b77a2eb8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:22.783357 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:22.784652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.784983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.785611 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] PATH_INFO: `/v3/projects/cb64ff08627b4823b3a4cd965b676c86/users/d2e91ed536b14e00b74b8dfaad7e7c5c/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.816017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb64ff08627b4823b3a4cd965b676c86, user_id=d2e91ed536b14e00b74b8dfaad7e7c5c, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:22.819574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:22.842561 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1363/2706] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:22 2022] PUT /identity/v3/projects/cb64ff08627b4823b3a4cd965b676c86/users/d2e91ed536b14e00b74b8dfaad7e7c5c/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:22.855464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:22.855751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:22.856024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:22.893863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] MFA Rules not processed for user `d2e91ed536b14e00b74b8dfaad7e7c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:22.895384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:22.995048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1344/2707] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:22.996632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.112174 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10c4d9-34b5-47dc-8a97-7797de2930a7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.118172 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.121211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.121211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.121211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.121211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.122537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.165356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1364/2708] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:22 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:23.169077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.181790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37708042-30e1-4779-9ccc-03720c9b6817 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.255354 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.256489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.256489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.256489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.257946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.259909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.284744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1345/2709] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:23 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:23.289348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.300250 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032fa4f-53ad-4253-a470-22173e0a0950 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.305929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.306564 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.306921 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.307518 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.308245 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.310001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.326322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1365/2710] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:23.330372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.341915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f39ba-6070-4af9-ba56-a0b5ddc7e971 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.346981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.347668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.348141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.348617 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] PATH_INFO: `/v3/projects/3b3e357917d547019233cd1e7c97da6d/users/6a6180c79f4f4ded8e92fa1c6ab50e10/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.365723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3e357917d547019233cd1e7c97da6d, user_id=6a6180c79f4f4ded8e92fa1c6ab50e10, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.369293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.384773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1346/2711] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:23 2022] PUT /identity/v3/projects/3b3e357917d547019233cd1e7c97da6d/users/6a6180c79f4f4ded8e92fa1c6ab50e10/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:49:23.388662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.400254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23828b5-243c-4c5e-bf29-063fff6c5c68 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.490162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.490837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.491109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.492125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.492798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.494517 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b7c1b-7944-436b-87af-3c1300f3e855 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.513363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1366/2712] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:23.517384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.537413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee892b-6490-45f7-a090-02de048a08aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.544350 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.545881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.546558 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.547261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] PATH_INFO: `/v3/projects/3b3e357917d547019233cd1e7c97da6d/users/6a6180c79f4f4ded8e92fa1c6ab50e10/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.560109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3e357917d547019233cd1e7c97da6d, user_id=6a6180c79f4f4ded8e92fa1c6ab50e10, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.563121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.595300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1347/2713] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:23 2022] PUT /identity/v3/projects/3b3e357917d547019233cd1e7c97da6d/users/6a6180c79f4f4ded8e92fa1c6ab50e10/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:23.606171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7073e78-a6fc-4b52-bed8-df532517bddc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.606462 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7073e78-a6fc-4b52-bed8-df532517bddc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.606742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7073e78-a6fc-4b52-bed8-df532517bddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.662603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a7073e78-a6fc-4b52-bed8-df532517bddc None None] MFA Rules not processed for user `6a6180c79f4f4ded8e92fa1c6ab50e10`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:23.664081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7073e78-a6fc-4b52-bed8-df532517bddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:23.685529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:23.798043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1367/2714] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:23.811457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:23.813418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:23.855298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:23.887409 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.887701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.888159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.901770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f31e8-0876-425b-af99-73d1bb78fbde admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:23.905748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:23.906723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:23.906723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:23.907012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:23.919507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:23.921139 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:23.932955 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1348/2715] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:23.934153 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] MFA Rules not processed for user `4496d5005b0040e8b70020740e9e3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:23.935392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:24.008119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1368/2716] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:24.031498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:24.033037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:24.140991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d99e20-e471-4799-8cc6-63632ce47d29 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:24.146772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:24.147418 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:24.148585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:24.148715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:24.156901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:24.158537 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:24.170317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1349/2717] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:24.394083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:24.409700 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf787-f1af-47f7-b368-3bbbbeaf2cf6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:24.415283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a6180c79f4f4ded8e92fa1c6ab50e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b3e357917d547019233cd1e7c97da6d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:24.416166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:24.416838 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:24.417432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:24.418228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:24.418771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:24.420803 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:24.440466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1369/2718] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:24.441757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:24.452489 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e691c3-33f8-48f2-93ad-d01eb468ed1b service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:24.457109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a6180c79f4f4ded8e92fa1c6ab50e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b3e357917d547019233cd1e7c97da6d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:24.457736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:24.458015 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:24.458287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:24.458911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:24.460836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:24.495769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1350/2719] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:24 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:24.499872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:24.512085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17af3a6-1d84-49a9-b408-d9645428174b tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:24.588825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a6180c79f4f4ded8e92fa1c6ab50e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b3e357917d547019233cd1e7c97da6d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:24.589491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:24.589856 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:24.590251 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:24.590971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:24.591343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:24.593184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:24.606849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1370/2720] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:24.650705 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:24.664253 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6fb48-ddb3-40be-8f7e-53cf5a580e61 tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:24.669087 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a6180c79f4f4ded8e92fa1c6ab50e10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b3e357917d547019233cd1e7c97da6d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:24.669992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:24.670339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:24.670685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] PATH_INFO: `/v3/projects/31f99b8db6054758bc61eaa64733a58d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:24.671908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorizing `identity:delete_project(project_id=31f99b8db6054758bc61eaa64733a58d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:24.673922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:24.734453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f99b8db6054758bc61eaa64733a58d', 'request_id': 'req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:24.743907 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f99b8db6054758bc61eaa64733a58d', 'request_id': 'req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:24.749925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f99b8db6054758bc61eaa64733a58d', 'request_id': 'req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:24.772426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] The token cache is being invalidate because project 31f99b8db6054758bc61eaa64733a58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:24.772755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:24.774083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e1229e-abd2-4d18-b5b8-5e9750cdb48f tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:24.775981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1351/2721] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:24 2022] DELETE /identity/v3/projects/31f99b8db6054758bc61eaa64733a58d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:25.005601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:25.007961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:25.040708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:25.096621 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a31917-c973-4abf-afca-75a02ad5029e tempest-VolumeQuotaClassesTest-1557739884 tempest-VolumeQuotaClassesTest-1557739884-project-admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:25.169850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:25.170515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:25.170775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:25.171053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] PATH_INFO: `/v3/users/d2e91ed536b14e00b74b8dfaad7e7c5c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:25.172477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e91ed536b14e00b74b8dfaad7e7c5c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:25.174325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:25.236521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e91ed536b14e00b74b8dfaad7e7c5c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:25.237441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e91ed536b14e00b74b8dfaad7e7c5c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:25.245750 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e91ed536b14e00b74b8dfaad7e7c5c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:25.246524 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e91ed536b14e00b74b8dfaad7e7c5c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:25.255753 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e91ed536b14e00b74b8dfaad7e7c5c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:25.276745 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1371/2722] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:25 2022] DELETE /identity/v3/users/d2e91ed536b14e00b74b8dfaad7e7c5c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:25.282016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d670fe9-c620-4401-a72b-cc5077b34bab None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:25.282016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d670fe9-c620-4401-a72b-cc5077b34bab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:25.282016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d670fe9-c620-4401-a72b-cc5077b34bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:25.305562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4d670fe9-c620-4401-a72b-cc5077b34bab None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:25.306742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d670fe9-c620-4401-a72b-cc5077b34bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:25.317902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:25.407168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1352/2723] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:25.417977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:25.419707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:25.497743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7c424-6ff8-45d0-a85c-c6122d90d6df admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:25.501653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:25.512981 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:25.514441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:25.515378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:25.518051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:25.530591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:25.532415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:25.551159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1372/2724] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:25.658717 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:25.659288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:25.659288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:25.682556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] MFA Rules not processed for user `4496d5005b0040e8b70020740e9e3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:25.684196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:25.772644 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1353/2725] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:49:25.921940 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:25.923433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:25.951165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dd228-bf79-4215-9128-5ccb64f04c02 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:25.960163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:25.960812 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:25.961090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:25.961390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] PATH_INFO: `/v3/projects/cb64ff08627b4823b3a4cd965b676c86` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:25.964678 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb64ff08627b4823b3a4cd965b676c86)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:25.966617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:26.027177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb64ff08627b4823b3a4cd965b676c86', 'request_id': 'req-6717feb7-cfbf-426f-b24b-e50d01506ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.039977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb64ff08627b4823b3a4cd965b676c86', 'request_id': 'req-6717feb7-cfbf-426f-b24b-e50d01506ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.049104 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb64ff08627b4823b3a4cd965b676c86', 'request_id': 'req-6717feb7-cfbf-426f-b24b-e50d01506ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.062408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] The token cache is being invalidate because project cb64ff08627b4823b3a4cd965b676c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:26.062794 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.063528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.068163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1373/2726] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:25 2022] DELETE /identity/v3/projects/cb64ff08627b4823b3a4cd965b676c86 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:26.069553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:26.083405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8238534-2f88-4762-ae56-ab79f83f6b7e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:26.147967 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:26.148799 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:26.149080 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:26.149401 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] PATH_INFO: `/v3/users/6a6180c79f4f4ded8e92fa1c6ab50e10` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:26.150597 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6180c79f4f4ded8e92fa1c6ab50e10)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:26.152609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:26.214067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6180c79f4f4ded8e92fa1c6ab50e10'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.222390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6180c79f4f4ded8e92fa1c6ab50e10'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.233386 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6180c79f4f4ded8e92fa1c6ab50e10'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.233994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6180c79f4f4ded8e92fa1c6ab50e10'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.235468 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:26.250758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6180c79f4f4ded8e92fa1c6ab50e10'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.261184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1354/2727] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:26 2022] DELETE /identity/v3/users/6a6180c79f4f4ded8e92fa1c6ab50e10 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:26.420511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:26.423204 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:26.513094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717feb7-cfbf-426f-b24b-e50d01506ec0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:26.568181 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:26.568814 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:26.569164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:26.569500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:26.571400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:26.582618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:26.584088 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539e8f6-6289-440e-aa60-4750b5a3bda1 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:26.595984 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1374/2728] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:26.604205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:26.605807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:26.718755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd722994-d94f-45e5-b708-5a6f252ecad7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:26.725034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:26.726026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:26.726373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:26.726770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] PATH_INFO: `/v3/projects/3b3e357917d547019233cd1e7c97da6d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:26.727842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3e357917d547019233cd1e7c97da6d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:26.729823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:26.790869 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3e357917d547019233cd1e7c97da6d', 'request_id': 'req-afdd968a-ec31-4fa5-b9e6-c06848c0049d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.798424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3e357917d547019233cd1e7c97da6d', 'request_id': 'req-afdd968a-ec31-4fa5-b9e6-c06848c0049d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.806406 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3e357917d547019233cd1e7c97da6d', 'request_id': 'req-afdd968a-ec31-4fa5-b9e6-c06848c0049d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.825825 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] The token cache is being invalidate because project 3b3e357917d547019233cd1e7c97da6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:26.826122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.826786 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-afdd968a-ec31-4fa5-b9e6-c06848c0049d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:26.828550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1355/2729] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:26 2022] DELETE /identity/v3/projects/3b3e357917d547019233cd1e7c97da6d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:26.833489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:26.833774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:26.834049 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:26.855440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:26.856987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:26.891596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:26.939156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1375/2730] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:26.947921 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:26.948202 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:26.948474 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:26.976827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:26.978090 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:26.987042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1356/2731] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:26.991851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.002519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6914e7-b1fd-44b1-93cb-2fac6e034024 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.017475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.018439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.018719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.018988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.019648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:27.019952 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.021681 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.035722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1376/2732] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 07:49:27.040121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.051037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeffca2-40ae-45e3-a66b-90d69540fe00 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.057478 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.058244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.058621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.059612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.060361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.062593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.098164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1357/2733] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:27 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:27.099413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.112866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a9b6e-aa8d-4a57-a968-0b169d876462 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.184281 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.185079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.185528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.185816 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.186425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.188494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.213516 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1377/2734] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:27 2022] POST /identity/v3/users => generated 304 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:27.217300 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.228714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f551b05-37fd-44f1-85cf-278bbe8ed50b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.233237 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.233845 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.234152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.234451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.235096 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.236879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.250320 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1358/2735] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:27.254322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.266506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd6eab-4dda-4824-8288-820b213aca10 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.271425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.272034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.272316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.272610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.273545 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.275227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1ae13-db23-4372-a945-f14765a2f7c8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.288768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1378/2736] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:27.294122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.310195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd6e5e-a64c-4ed8-a023-31298416bb7e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.315568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.316243 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.316901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.316901 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] PATH_INFO: `/v3/projects/b99800d74b15476393529539c3209019/users/1fc6515be3f94e3eb2d05313bb6d40cc/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.333298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99800d74b15476393529539c3209019, user_id=1fc6515be3f94e3eb2d05313bb6d40cc, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.336094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.351578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1359/2737] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:27 2022] PUT /identity/v3/projects/b99800d74b15476393529539c3209019/users/1fc6515be3f94e3eb2d05313bb6d40cc/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:27.363184 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.363632 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.364014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.403779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] MFA Rules not processed for user `1fc6515be3f94e3eb2d05313bb6d40cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:27.405068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:27.501669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1379/2738] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:27.506099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.527624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:27.611754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee649d-869c-4740-b6cc-9e60bc4b4bad admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.617409 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.618185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.618337 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.618700 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.619291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.621346 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.661684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1360/2739] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:27 2022] POST /identity/v3/projects => generated 407 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:27.666055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.678429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7606-eae9-4a31-af01-4b634f19d129 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.707131 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:27.742375 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.743000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.743284 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.743559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.744199 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.746175 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.779028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1380/2740] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:27 2022] POST /identity/v3/users => generated 310 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:27.782937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.800250 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c49e2-3d3f-4a5c-871e-9b3721ca20d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.805518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.806135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.806445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.806723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.807372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.809017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.820672 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1361/2741] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:27.824947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.835834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3199ec4-a063-435a-bcb0-37fe783bf95c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.841391 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.842096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.842618 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.842789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] PATH_INFO: `/v3/projects/8ef6311389614213bbf7aea0ea293e6d/users/c816683b1ce04842a70aa49d1221e1cb/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.861091 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef6311389614213bbf7aea0ea293e6d, user_id=c816683b1ce04842a70aa49d1221e1cb, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.863784 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.880290 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1381/2742] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:27 2022] PUT /identity/v3/projects/8ef6311389614213bbf7aea0ea293e6d/users/c816683b1ce04842a70aa49d1221e1cb/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:27.885425 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.896437 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dabb2-fef8-4965-a680-3e6addd054bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.961809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.962580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.962926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.963265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:27.964007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:27.965783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00601f76-36ad-4533-93ec-3c810c47dc84 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:27.977922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1362/2743] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:27.982227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:27.993368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9799f8-0067-4277-83e7-596dac533477 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:27.998211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:27.998891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:27.999171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:27.999445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] PATH_INFO: `/v3/projects/8ef6311389614213bbf7aea0ea293e6d/users/c816683b1ce04842a70aa49d1221e1cb/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:28.001604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef6311389614213bbf7aea0ea293e6d, user_id=c816683b1ce04842a70aa49d1221e1cb, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:28.004265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:28.017592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1382/2744] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:27 2022] PUT /identity/v3/projects/8ef6311389614213bbf7aea0ea293e6d/users/c816683b1ce04842a70aa49d1221e1cb/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:49:28.028866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:28.029181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:28.029467 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:28.060488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] MFA Rules not processed for user `c816683b1ce04842a70aa49d1221e1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:28.061770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:28.130954 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1363/2745] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:28 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:28.142586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:28.144296 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:28.205326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f3317-6834-4efa-b1a8-ebf1d522ea8c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:28.208885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:28.209512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:28.209786 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:28.210648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:28.219702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:28.221137 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:28.230390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1383/2746] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:28 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:28.392812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:28.398302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:28.404992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a369aa-0d35-43e8-a44b-ce14a69ec8d6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:28.409274 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:28.409880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:28.410160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:28.410434 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:28.470776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:28.472145 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:28.480913 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1364/2747] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:28 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:31.931686 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:31.942749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bff6ab-b899-42f6-a8ed-5d4b425ce201 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:31.946506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:31.947276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:31.947507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:31.947779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:31.949548 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:32.019883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:32.022034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:32.031665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1384/2748] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:38.476900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=27 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:42.027303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:42.079771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:42.116118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:42.118300 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:42.148446 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df705b8-800d-4972-9a6f-c7aba7abdde1 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:42.154773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:42.156028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.156311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.156585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] PATH_INFO: `/v3/users/1fc6515be3f94e3eb2d05313bb6d40cc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.158390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc6515be3f94e3eb2d05313bb6d40cc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:42.161695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:42.185915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aae578-2afd-4b14-bdd2-e7eb8ce086fe service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:42.190988 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:42.191747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.192085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.192469 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.194500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:42.217163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc6515be3f94e3eb2d05313bb6d40cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.225296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc6515be3f94e3eb2d05313bb6d40cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.233001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc6515be3f94e3eb2d05313bb6d40cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.233694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc6515be3f94e3eb2d05313bb6d40cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.253747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-591ce272-04e8-4592-bc3a-4cda8f193c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc6515be3f94e3eb2d05313bb6d40cc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.261204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1365/2749] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:42 2022] DELETE /identity/v3/users/1fc6515be3f94e3eb2d05313bb6d40cc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:42.262402 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:42.264219 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:42.265287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9353327-23df-4708-a5c1-784315e92310 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.265575 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9353327-23df-4708-a5c1-784315e92310 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.265875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9353327-23df-4708-a5c1-784315e92310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.274645 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1385/2750] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:42.288422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e9353327-23df-4708-a5c1-784315e92310 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:42.289772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9353327-23df-4708-a5c1-784315e92310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:42.360917 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1366/2751] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:49:42.370860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:42.382719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dda682-e050-41ce-8af5-d3a1b31337da service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:42.445351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:42.447213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.447498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.447777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.460968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:42.463070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:42.474853 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1386/2752] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:42.770102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9353327-23df-4708-a5c1-784315e92310 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:42.771840 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9353327-23df-4708-a5c1-784315e92310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:42.786099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9353327-23df-4708-a5c1-784315e92310 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:42.802164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:42.802164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.802164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.802164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] PATH_INFO: `/v3/projects/b99800d74b15476393529539c3209019` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.802510 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99800d74b15476393529539c3209019)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:42.810653 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:42.829232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:42.862594 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99800d74b15476393529539c3209019', 'request_id': 'req-637ac9d5-440e-4793-83e7-7ec08f5d6db1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.869935 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99800d74b15476393529539c3209019', 'request_id': 'req-637ac9d5-440e-4793-83e7-7ec08f5d6db1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.875915 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99800d74b15476393529539c3209019', 'request_id': 'req-637ac9d5-440e-4793-83e7-7ec08f5d6db1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.889541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] The token cache is being invalidate because project b99800d74b15476393529539c3209019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:42.889833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.891070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:42.894679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1367/2753] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:42 2022] DELETE /identity/v3/projects/b99800d74b15476393529539c3209019 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:42.898659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:42.909463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96be901-a9c4-4438-970f-59304422b0cd service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:42.971127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:42.980238 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:42.980872 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:42.981183 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:42.981463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] PATH_INFO: `/v3/users/c816683b1ce04842a70aa49d1221e1cb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:42.982915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c816683b1ce04842a70aa49d1221e1cb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:42.984650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.035658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c816683b1ce04842a70aa49d1221e1cb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.036325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c816683b1ce04842a70aa49d1221e1cb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.043572 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c816683b1ce04842a70aa49d1221e1cb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.044211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c816683b1ce04842a70aa49d1221e1cb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.052847 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec6c90c6-27d6-49e9-a2ce-f2531367f6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c816683b1ce04842a70aa49d1221e1cb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.073306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1387/2754] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:49:42 2022] DELETE /identity/v3/users/c816683b1ce04842a70aa49d1221e1cb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:43.346792 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.348445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:43.365221 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ac9d5-440e-4793-83e7-7ec08f5d6db1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.430475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.431120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.431426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.431694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] PATH_INFO: `/v3/projects/8ef6311389614213bbf7aea0ea293e6d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.432863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef6311389614213bbf7aea0ea293e6d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.434850 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.490560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef6311389614213bbf7aea0ea293e6d', 'request_id': 'req-b078329e-0969-4557-ae9e-5f66eae9bb8a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.499181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef6311389614213bbf7aea0ea293e6d', 'request_id': 'req-b078329e-0969-4557-ae9e-5f66eae9bb8a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.505348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef6311389614213bbf7aea0ea293e6d', 'request_id': 'req-b078329e-0969-4557-ae9e-5f66eae9bb8a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.518762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] The token cache is being invalidate because project 8ef6311389614213bbf7aea0ea293e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:49:43.519054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.519699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b078329e-0969-4557-ae9e-5f66eae9bb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:49:43.521256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1368/2755] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:49:43 2022] DELETE /identity/v3/projects/8ef6311389614213bbf7aea0ea293e6d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:43.527156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.527437 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.527747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.548954 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:43.550201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:43.611279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1388/2756] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:43.619937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.620307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.620656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.641045 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:43.642361 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:43.651781 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1369/2757] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:43.656085 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.666259 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872591b-44b2-402a-9abf-ba616304abbd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.670883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.671479 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.672115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.672115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.672641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:49:43.672939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.674578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.686066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1389/2758] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:49:43.690518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.699980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cf89a-4cda-476e-9a41-148f8e7525aa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.704828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.705476 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.705802 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.706081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.706729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.708721 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.739872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1370/2759] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:43 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:43.743990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.755634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff3889-fb7b-4cb3-9dbe-ce4883504426 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.846332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.846970 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.847254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.847667 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.848497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.850577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.860610 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:43.880520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1390/2760] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:43 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:43.885380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.893970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:43.899549 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2d545-9bde-4e47-838e-c5c3c57edeae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.906241 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.906981 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.907900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.907900 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.908223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.910842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.929370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.939599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1371/2761] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:43.941998 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31afb532-33b8-4282-b803-6ae64eca6223 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.947871 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.948619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.948907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.950211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:43.950211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:43.952198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2855611-fc12-45eb-92ac-a12afd2a19d3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:43.969660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:43.970582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1391/2762] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:43.980423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e706254-b91d-4b75-8777-047c9282dc46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:43.985820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:43.986580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:43.986875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:43.987156 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] PATH_INFO: `/v3/projects/933739f271fd4cfd93e9b933808a1420/users/0057e0e07d62424883eabaa3332ff65e/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:44.008609 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933739f271fd4cfd93e9b933808a1420, user_id=0057e0e07d62424883eabaa3332ff65e, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:44.012505 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:44.033545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1372/2763] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:43 2022] PUT /identity/v3/projects/933739f271fd4cfd93e9b933808a1420/users/0057e0e07d62424883eabaa3332ff65e/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:44.038874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4f2c-9a09-4a6d-ac8f-23d30ea4ecdb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:44.038874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4f2c-9a09-4a6d-ac8f-23d30ea4ecdb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:44.039510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4f2c-9a09-4a6d-ac8f-23d30ea4ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:44.061179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c19e4f2c-9a09-4a6d-ac8f-23d30ea4ecdb None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:44.062631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c19e4f2c-9a09-4a6d-ac8f-23d30ea4ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:44.192630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1392/2764] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:44.208038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:44.211054 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:44.284506 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bd35f-ec1b-4e71-9e03-b91c0366bbc2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:44.288291 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:44.288902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:44.289267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:44.289548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:44.301384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:44.302946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:44.313798 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1373/2765] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:45.089741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.090203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.090299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.122953 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] MFA Rules not processed for user `0057e0e07d62424883eabaa3332ff65e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:45.124191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:45.195000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1393/2766] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:45.199352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.210339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec6c4-40f8-4e7d-86d5-cf8dcc3b5d80 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.216217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.216815 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.217222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.217593 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.218305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.220325 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.255222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1374/2767] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:49:45 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:45.256429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.271426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c9758-557f-40a5-b3d0-6519eb637e98 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.341034 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.341876 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.342252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.342604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.343307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.345679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.389103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:45.402317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1394/2768] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:49:45 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:49:45.403908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.421547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5edbe-6805-417a-bed5-48137a67f509 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.433883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.434785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.434785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.435048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.435706 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.437420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.460689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1375/2769] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:49:45.464851 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.476915 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec47e81-680f-4f56-a9c8-009bd74ba1cf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.482247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.483004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.483298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.483648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] PATH_INFO: `/v3/projects/f74dff8978f44af99ef763efc70426e5/users/b6ce8b3e7709470785dde12f8ab2aa02/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.511283 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74dff8978f44af99ef763efc70426e5, user_id=b6ce8b3e7709470785dde12f8ab2aa02, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.514203 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.550021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1395/2770] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:45 2022] PUT /identity/v3/projects/f74dff8978f44af99ef763efc70426e5/users/b6ce8b3e7709470785dde12f8ab2aa02/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:49:45.552083 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.562947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9a64-71bc-490b-8922-bc689d5e582a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.636130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.637108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.637632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.637976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.638992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.640699 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6691e-776c-4f3d-9c28-5fb8ea7ea4a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.657722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1376/2771] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:49:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:49:45.658089 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:45.670254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847aa98-a170-4e42-a03e-a16e70e177e9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:45.675070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:45.675926 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:45.676231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:45.676506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] PATH_INFO: `/v3/projects/f74dff8978f44af99ef763efc70426e5/users/b6ce8b3e7709470785dde12f8ab2aa02/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:45.679404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74dff8978f44af99ef763efc70426e5, user_id=b6ce8b3e7709470785dde12f8ab2aa02, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:45.682828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:45.694069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1396/2772] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:49:45 2022] PUT /identity/v3/projects/f74dff8978f44af99ef763efc70426e5/users/b6ce8b3e7709470785dde12f8ab2aa02/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:49:46.334951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:46.335436 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:46.335548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:46.363560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] MFA Rules not processed for user `b6ce8b3e7709470785dde12f8ab2aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:49:46.364769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:46.389256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:46.436035 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1377/2773] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:49:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:49:46.448074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:46.449804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:46.510724 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b837d-cbb6-489a-bc5c-35732f366f06 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:46.514396 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:46.515017 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:46.515294 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:46.515565 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:46.576733 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:46.578222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:46.588323 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1397/2774] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:47.982754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:49:47.984189 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:49:48.005975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff96bb-0402-4f7e-a79e-e4ea8f8815df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:49:48.009990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:49:48.010826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:49:48.011303 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:49:48.011787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:49:48.021400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:49:48.022766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:49:48.033422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1378/2775] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:49:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:49:56.583328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:49:58.028975 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=24 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:05.203265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:05.204693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:05.274881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:05.285877 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c3822-b005-4f87-86d5-b30a099e2375 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:05.291165 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:05.291874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:05.292156 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:05.292429 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:05.294637 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:05.305575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:05.307671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:05.317344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1398/2776] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:05.338259 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ba88e-ebf9-489b-bd5e-3e8b3a53a4f6 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:05.342686 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:05.343317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:05.343604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:05.343879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] PATH_INFO: `/v3/users/0057e0e07d62424883eabaa3332ff65e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:05.344994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0057e0e07d62424883eabaa3332ff65e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:05.347000 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:05.392306 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0057e0e07d62424883eabaa3332ff65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.399288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0057e0e07d62424883eabaa3332ff65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.405999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0057e0e07d62424883eabaa3332ff65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.406805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0057e0e07d62424883eabaa3332ff65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.418588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0057e0e07d62424883eabaa3332ff65e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.425069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1379/2777] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:50:05 2022] DELETE /identity/v3/users/0057e0e07d62424883eabaa3332ff65e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:05.606833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:05.618302 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5595be1-7893-42dd-a49a-8848c41198f9 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:05.681198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:05.682426 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:05.682703 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:05.682973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] PATH_INFO: `/v3/projects/933739f271fd4cfd93e9b933808a1420` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:05.684081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933739f271fd4cfd93e9b933808a1420)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:05.686012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:05.730617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933739f271fd4cfd93e9b933808a1420', 'request_id': 'req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.737320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933739f271fd4cfd93e9b933808a1420', 'request_id': 'req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.742898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933739f271fd4cfd93e9b933808a1420', 'request_id': 'req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.754528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] The token cache is being invalidate because project 933739f271fd4cfd93e9b933808a1420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:50:05.754819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.755460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.757021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1399/2778] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:50:05 2022] DELETE /identity/v3/projects/933739f271fd4cfd93e9b933808a1420 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:05.761192 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:05.762688 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:05.776066 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c93c7-adc3-4196-b729-523f92bfb61b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:05.831471 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:05.832122 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:05.832395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:05.832690 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] PATH_INFO: `/v3/users/b6ce8b3e7709470785dde12f8ab2aa02` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:05.833847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ce8b3e7709470785dde12f8ab2aa02)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:05.835828 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:05.857553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:05.884181 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ce8b3e7709470785dde12f8ab2aa02'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.891196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ce8b3e7709470785dde12f8ab2aa02'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.898922 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ce8b3e7709470785dde12f8ab2aa02'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.899544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ce8b3e7709470785dde12f8ab2aa02'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.912934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c8b760f-0ddd-4cbe-83fa-f12b5b8bb27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ce8b3e7709470785dde12f8ab2aa02'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:05.919572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1380/2779] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:50:05 2022] DELETE /identity/v3/users/b6ce8b3e7709470785dde12f8ab2aa02 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:06.090316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.091744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:06.105009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906c0a8-84b2-4dff-8db8-215cf9b4ec25 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.121820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:06.164380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.165003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.165322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.166132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] PATH_INFO: `/v3/projects/f74dff8978f44af99ef763efc70426e5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.167257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74dff8978f44af99ef763efc70426e5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.169182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.213285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74dff8978f44af99ef763efc70426e5', 'request_id': 'req-afa6f868-d15a-4aa7-ab99-a11d279b9880'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:06.220031 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74dff8978f44af99ef763efc70426e5', 'request_id': 'req-afa6f868-d15a-4aa7-ab99-a11d279b9880'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:06.226122 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74dff8978f44af99ef763efc70426e5', 'request_id': 'req-afa6f868-d15a-4aa7-ab99-a11d279b9880'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:06.237446 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] The token cache is being invalidate because project f74dff8978f44af99ef763efc70426e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:50:06.237732 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:06.238363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-afa6f868-d15a-4aa7-ab99-a11d279b9880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:06.239824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1400/2780] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:50:06 2022] DELETE /identity/v3/projects/f74dff8978f44af99ef763efc70426e5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:06.244727 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.245018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.245321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.265286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:50:06.266771 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:06.323650 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1381/2781] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:50:06.331316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.331606 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.331918 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.352269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:50:06.353519 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:06.361747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1401/2782] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:50:06.365889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.375548 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07dcefa-21c7-4598-9109-c91ec12dbcea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.380135 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.380732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.381010 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.381323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.381966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:50:06.382266 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.383858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.395546 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1382/2783] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:50:06.399870 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.409494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea67fc-0d32-41f4-b953-795373695194 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.414078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.414663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.414941 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.415210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.415850 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.417763 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.443929 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1402/2784] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:50:06.448277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.458028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bee38-419c-4093-a133-ea02ee48dc3a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.513703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.514233 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.514513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.514785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.515400 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.517335 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.538651 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1383/2785] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:50:06.542744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.552882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a21cd-dbee-4c43-aa58-f1911a19be82 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.557508 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.558096 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.558373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.558646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.559279 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.560875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.570906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1403/2786] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:50:06.575305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.585854 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9060704-a482-488a-b996-17469c0bf512 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.591121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.591715 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.591992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.592263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.592894 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.594516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93e378-8ab2-40fa-b1b8-dcaf0197e384 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.604920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1384/2787] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:50:06.608951 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.620148 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca356164-fde9-440a-9a1d-bd39fd3a3986 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.624747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.625408 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.625698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.626150 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] PATH_INFO: `/v3/projects/689f73a000b34ae583d9da020ce268b9/users/d1d9ddc292da40d98e6b8bf2559252c8/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.641169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689f73a000b34ae583d9da020ce268b9, user_id=d1d9ddc292da40d98e6b8bf2559252c8, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.643806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.657472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1404/2788] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:50:06 2022] PUT /identity/v3/projects/689f73a000b34ae583d9da020ce268b9/users/d1d9ddc292da40d98e6b8bf2559252c8/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:06.667768 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.668051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.668323 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.677212 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:06.699530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] MFA Rules not processed for user `d1d9ddc292da40d98e6b8bf2559252c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:50:06.700774 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:06.772629 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1385/2789] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:50:06.780617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.829338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:06.845514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f2afa-0505-4008-8300-936c8f35eaa4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.850198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.850790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.851073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.851343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.851973 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.853881 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.880473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1405/2790] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:50:06.885582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.896069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050aa66-52b0-40bb-a97b-c19fdc9a8707 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.952703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.953356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.953633 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.953902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:06.954531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:06.956451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:06.978142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1386/2791] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:50:06 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:50:06.982369 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:06.993002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e20df-7bb9-4cfa-93b1-f4a5d161f16d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:06.998222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:06.998805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:06.999079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:06.999351 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.000067 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:07.001925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:07.014213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1406/2792] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:50:07.018299 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:07.028950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ba6f5-9d38-47d6-9d49-97753b173c58 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:07.034486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:07.035440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:07.035723 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:07.035995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] PATH_INFO: `/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c/users/1a473be8581349c085be83a258f744b6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.053501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b62cf8c56d24ebb8554dda70e59a91c, user_id=1a473be8581349c085be83a258f744b6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:07.056725 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:07.069926 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1387/2793] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:50:07 2022] PUT /identity/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c/users/1a473be8581349c085be83a258f744b6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:07.073774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:07.083864 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79d65-e6b5-4697-b716-cdcca3521868 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:07.141338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:07.141930 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:07.142211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:07.142483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.143138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:07.144734 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1de889-ec68-40d1-8595-85f4b537fd85 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:07.154711 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1407/2794] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:50:07.158814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:07.169520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc19074-a1ef-4ab3-9a93-2e9b73f5a9aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:07.174147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:07.174790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:07.175067 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:07.175341 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] PATH_INFO: `/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c/users/1a473be8581349c085be83a258f744b6/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.177267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b62cf8c56d24ebb8554dda70e59a91c, user_id=1a473be8581349c085be83a258f744b6, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:07.179890 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:07.190501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1388/2795] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:50:07 2022] PUT /identity/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c/users/1a473be8581349c085be83a258f744b6/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:07.201460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:07.201768 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:07.202074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.230021 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] MFA Rules not processed for user `1a473be8581349c085be83a258f744b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:50:07.231220 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:07.298445 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1408/2796] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:50:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:50:07.309754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:07.311458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:07.362554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:07.369351 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daefc5-21a9-453f-879f-17a1be592a73 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:07.373204 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:07.373794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:07.374068 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:07.374333 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:07.429187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:07.430656 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:07.439461 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1389/2797] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:11.648079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:11.658018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b016c2f-e637-46cf-8f06-8deb19dd3db5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:11.661520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:11.662118 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:11.662390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:11.662765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:11.671971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:11.673416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:11.683209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1409/2798] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:15.701560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:15.702972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:15.760534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe7dc4-5df2-46d9-a6e4-14e9e73054b7 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:15.764722 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:15.765363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:15.765652 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:15.765925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:15.767664 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:15.831818 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:15.833281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:15.843711 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1390/2799] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:16.362934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:16.364510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:16.378689 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f9ae6-28ed-4483-8f9f-c38aa5f1b78b service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:16.383244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:16.383860 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:16.384142 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:16.384416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] PATH_INFO: `/v3/users/c6a74f33e6e941ed8c3f7850bdf29bb0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:16.386277 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a74f33e6e941ed8c3f7850bdf29bb0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:16.388191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:16.411078 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:16.436577 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a74f33e6e941ed8c3f7850bdf29bb0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.437269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a74f33e6e941ed8c3f7850bdf29bb0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.445066 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a74f33e6e941ed8c3f7850bdf29bb0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.445791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a74f33e6e941ed8c3f7850bdf29bb0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.453694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a74f33e6e941ed8c3f7850bdf29bb0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.470720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1410/2800] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:50:16 2022] DELETE /identity/v3/users/c6a74f33e6e941ed8c3f7850bdf29bb0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:16.538466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:16.548322 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172a00d-3463-4060-bcd5-70ec6a87da64 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:16.603488 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:16.614396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:16.615044 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:16.615324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:16.615600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] PATH_INFO: `/v3/users/d1d9ddc292da40d98e6b8bf2559252c8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:16.616751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d9ddc292da40d98e6b8bf2559252c8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:16.618735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:16.651705 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:16.653348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:16.668521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa807653-7dd0-4387-b07a-880083f3c076 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:16.674710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:16.675582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:16.675868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:16.676147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] PATH_INFO: `/v3/projects/7170a1232ab14cd4aff705e91364d5c1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:16.677328 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7170a1232ab14cd4aff705e91364d5c1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:16.679319 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:16.682829 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d9ddc292da40d98e6b8bf2559252c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.690051 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d9ddc292da40d98e6b8bf2559252c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.698611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d9ddc292da40d98e6b8bf2559252c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.699402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d9ddc292da40d98e6b8bf2559252c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.712554 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d6e38bb-ccbd-4044-ba06-e996153b9303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d9ddc292da40d98e6b8bf2559252c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.719276 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1391/2801] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:50:16 2022] DELETE /identity/v3/users/d1d9ddc292da40d98e6b8bf2559252c8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:16.723663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:16.723895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:16.724173 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:16.729456 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7170a1232ab14cd4aff705e91364d5c1', 'request_id': 'req-bb5aee93-a664-443f-914b-7b964537793c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.737772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7170a1232ab14cd4aff705e91364d5c1', 'request_id': 'req-bb5aee93-a664-443f-914b-7b964537793c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.743037 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:50:16.743650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7170a1232ab14cd4aff705e91364d5c1', 'request_id': 'req-bb5aee93-a664-443f-914b-7b964537793c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.744980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:16.758769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] The token cache is being invalidate because project 7170a1232ab14cd4aff705e91364d5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:50:16.759060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.759859 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:16.762260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1411/2802] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:50:16 2022] DELETE /identity/v3/projects/7170a1232ab14cd4aff705e91364d5c1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:50:16.814331 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1392/2803] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 07:50:16.824523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:16.825990 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:16.889807 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5aee93-a664-443f-914b-7b964537793c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:16.944743 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:16.959822 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:16.960430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:16.960710 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:16.960983 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:16.970072 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:16.971731 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:16.982069 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1412/2804] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:17.246215 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:17.248032 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:17.262783 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4914c-c99a-4032-804d-ff4cdfbd031e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:17.269047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:17.269911 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:17.270200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:17.270503 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] PATH_INFO: `/v3/projects/689f73a000b34ae583d9da020ce268b9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:17.271698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689f73a000b34ae583d9da020ce268b9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:17.274060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:17.320619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689f73a000b34ae583d9da020ce268b9', 'request_id': 'req-9b157e8a-a52d-47ff-9bd3-912a5a088479'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.326680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689f73a000b34ae583d9da020ce268b9', 'request_id': 'req-9b157e8a-a52d-47ff-9bd3-912a5a088479'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.332110 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689f73a000b34ae583d9da020ce268b9', 'request_id': 'req-9b157e8a-a52d-47ff-9bd3-912a5a088479'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.334528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:17.346370 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] The token cache is being invalidate because project 689f73a000b34ae583d9da020ce268b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:50:17.346669 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.347328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.348833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1393/2805] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:50:17 2022] DELETE /identity/v3/projects/689f73a000b34ae583d9da020ce268b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:17.352852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:17.362961 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbca1e4-4ef0-4269-9f76-536e887a4b42 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:17.417624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:17.418244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:17.418520 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:17.418789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] PATH_INFO: `/v3/users/1a473be8581349c085be83a258f744b6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:17.419885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a473be8581349c085be83a258f744b6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:17.421829 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:17.465493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a473be8581349c085be83a258f744b6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.466140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a473be8581349c085be83a258f744b6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.472975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a473be8581349c085be83a258f744b6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.473625 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a473be8581349c085be83a258f744b6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.482008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a473be8581349c085be83a258f744b6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.500459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1413/2806] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:50:17 2022] DELETE /identity/v3/users/1a473be8581349c085be83a258f744b6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:17.760366 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:17.763169 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:17.777528 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b157e8a-a52d-47ff-9bd3-912a5a088479 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:17.833324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:17.833947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:17.834223 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:17.834494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] PATH_INFO: `/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:17.835585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b62cf8c56d24ebb8554dda70e59a91c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:17.837547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:17.885013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b62cf8c56d24ebb8554dda70e59a91c', 'request_id': 'req-f6c80de7-6c62-4874-a3e1-f83fc06203ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.891648 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b62cf8c56d24ebb8554dda70e59a91c', 'request_id': 'req-f6c80de7-6c62-4874-a3e1-f83fc06203ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.897036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b62cf8c56d24ebb8554dda70e59a91c', 'request_id': 'req-f6c80de7-6c62-4874-a3e1-f83fc06203ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.908872 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] The token cache is being invalidate because project 0b62cf8c56d24ebb8554dda70e59a91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:50:17.909191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.909806 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:50:17.911282 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1394/2807] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:50:17 2022] DELETE /identity/v3/projects/0b62cf8c56d24ebb8554dda70e59a91c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:50:27.496801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:27.906013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:38.858276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:38.859701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:38.927827 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302dfbc-c834-4296-9352-406571ddb37e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:38.974309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:38.974908 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:38.975186 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:38.975459 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:38.977257 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:38.988540 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:38.990040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:38.998332 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1414/2808] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:50:45.184634 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:50:45.186097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:45.198125 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80de7-6c62-4874-a3e1-f83fc06203ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:50:45.202312 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:50:45.202902 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:50:45.203180 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:50:45.203453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:50:45.205107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:50:45.274420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:50:45.275891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52848e1-ce30-406e-95e0-9aea1ede5ee1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:50:45.286755 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1395/2809] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:50:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:50:48.994424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=51 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:50:55.281851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=34 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:51:49.724628 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:51:49.726055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:51:49.743352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c47078-2177-4e7d-a229-8c8285833efb service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:51:49.749514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:51:49.750099 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:51:49.750377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:51:49.750649 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:51:49.752350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:51:49.810834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:51:49.812487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f48ae-8127-415c-b103-a9bb0425446b service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:51:49.821444 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1415/2810] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:51:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:51:54.590849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:51:54.591326 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:51:54.591451 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:51:54.611157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] MFA Rules not processed for user `93b2cf587dc44209a186fc453ac5bdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:51:54.612348 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:51:54.619157 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1396/2811] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:51:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:51:59.817260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=28 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:52:04.605029 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:52:28.534948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:52:28.535496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:52:28.535496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:52:28.555071 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] MFA Rules not processed for user `4496d5005b0040e8b70020740e9e3401`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:52:28.556301 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:52:28.612023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1416/2812] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:52:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:52:38.605366 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:52:45.154635 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:52:45.156490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:52:45.281892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4594f5-1124-4148-85c3-6dc7754a2148 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:52:45.287567 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:52:45.288187 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:52:45.288464 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:52:45.288763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:52:45.290588 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:52:45.303287 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:52:45.304891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:52:45.315920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1397/2813] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:52:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:52:55.310758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=52 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:53:55.272045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:55.273646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:55.299662 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad63344-ea98-4e01-825d-6e99da386422 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:55.304947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:55.305673 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:55.306027 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:55.306368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:55.308487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:55.323785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:55.325341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:55.333297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:55.335115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:55.338287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1417/2814] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:53:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:53:55.418191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ae0ee-1103-4a5f-89ee-50fec3f983ea service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:55.422738 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:55.423433 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:55.423718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:55.423994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] PATH_INFO: `/v3/users/4496d5005b0040e8b70020740e9e3401` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:55.425185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4496d5005b0040e8b70020740e9e3401)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:55.427137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:55.477206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4496d5005b0040e8b70020740e9e3401'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.485256 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4496d5005b0040e8b70020740e9e3401'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.494857 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4496d5005b0040e8b70020740e9e3401'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.495475 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4496d5005b0040e8b70020740e9e3401'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.512816 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ed41ce66-a8a1-4f9c-b643-17e71bda9610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4496d5005b0040e8b70020740e9e3401'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.518720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1398/2815] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:53:55 2022] DELETE /identity/v3/users/4496d5005b0040e8b70020740e9e3401 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:53:55.707790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:55.719694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3381759-6df9-4ab7-a891-374c264d11c9 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:55.782591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:55.784127 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:55.784509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:55.784925 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] PATH_INFO: `/v3/projects/6a0433dc28924fc891b5e114c521bb57` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:55.786342 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0433dc28924fc891b5e114c521bb57)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:55.788377 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:55.840569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0433dc28924fc891b5e114c521bb57', 'request_id': 'req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.847650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0433dc28924fc891b5e114c521bb57', 'request_id': 'req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.853491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0433dc28924fc891b5e114c521bb57', 'request_id': 'req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.866505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] The token cache is being invalidate because project 6a0433dc28924fc891b5e114c521bb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:53:55.866817 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.867507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f6fbf2cf-8a76-4aff-9a1d-8e80f018ce7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:53:55.869227 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1418/2816] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:53:55 2022] DELETE /identity/v3/projects/6a0433dc28924fc891b5e114c521bb57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:53:55.874849 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:55.875132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:55.875405 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:55.895659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:53:55.897038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:55.968086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1399/2817] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:53:55.977235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:55.977526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:55.977825 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:55.998241 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:53:55.999472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:56.008117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1419/2818] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:53:56.012399 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.021987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1bbe4-eb71-428d-be54-8d34acfb59cc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.026883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.027483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.027763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.028034 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.028681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:53:56.028987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.030612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.041966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1400/2819] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:53:56.046030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.055800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8285-44bf-42b1-af32-43aed6a081cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.060591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.061212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.061582 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.061805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.062380 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.064263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.091615 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1420/2820] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:53:56 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:53:56.095659 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.105532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e4d8b-7da1-45ff-9123-117af6ed1010 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.109070 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:53:56.162999 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.163601 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.163879 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.164147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.164764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.166695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.187174 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1401/2821] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:53:56 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:53:56.192119 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.202393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ac22-11ed-42f8-9092-591a06792ee4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.206942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.207526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.207800 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.208070 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.208704 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.210316 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.219965 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1421/2822] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:53:56.223676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.233550 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76437-740b-426d-b66d-06d55866dac9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.238666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.239254 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.239533 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.239803 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.240432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.242098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92b8d-ba2b-4c7b-b37b-a178759ef1d5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.254513 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1402/2823] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:53:56.258585 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.268943 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb00f-74bd-417a-84fa-f39745657a97 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.273605 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.274558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.274558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.274833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] PATH_INFO: `/v3/projects/462cfcb8aad746289ff3c2a01219da07/users/135b44145a1d4712b0ea9b3f00d1bc30/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.290779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=462cfcb8aad746289ff3c2a01219da07, user_id=135b44145a1d4712b0ea9b3f00d1bc30, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.293432 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.306535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1422/2824] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:53:56 2022] PUT /identity/v3/projects/462cfcb8aad746289ff3c2a01219da07/users/135b44145a1d4712b0ea9b3f00d1bc30/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:53:56.311581 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e3945-201d-4927-803b-6f154b9c5bd2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.311865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e3945-201d-4927-803b-6f154b9c5bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.312161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e3945-201d-4927-803b-6f154b9c5bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.450262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0f1e3945-201d-4927-803b-6f154b9c5bd2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:53:56.451487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f1e3945-201d-4927-803b-6f154b9c5bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:56.509027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1403/2825] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:53:56.518461 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:53:56.520157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:53:56.526747 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:53:56.577261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b6f03-83ce-4dec-9346-da18d973f305 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:53:56.580765 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:53:56.581393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:53:56.581669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:53:56.581956 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:53:56.590718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:53:56.592139 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:53:56.601996 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1423/2826] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:00.538447 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f115ec-e0cd-4792-92c1-7e2009bcc089 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:00.539568 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f115ec-e0cd-4792-92c1-7e2009bcc089 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:00.540055 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f115ec-e0cd-4792-92c1-7e2009bcc089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:00.574313 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e6f115ec-e0cd-4792-92c1-7e2009bcc089 None None] MFA Rules not processed for user `135b44145a1d4712b0ea9b3f00d1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:54:00.575800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e6f115ec-e0cd-4792-92c1-7e2009bcc089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:00.678224 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1404/2827] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:54:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:54:00.686912 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:00.688381 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:00.815010 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af87cf8-12d6-4a6f-a0a0-697c1f3b854c service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:00.821024 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:00.821659 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:00.821934 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:00.822207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:00.831176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:00.833423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419b27f-d339-4496-b2a8-c76c4dbe6ba0 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:00.843162 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1424/2828] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:02.120927 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:02.121479 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:02.121578 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:02.140810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] MFA Rules not processed for user `135b44145a1d4712b0ea9b3f00d1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:54:02.142074 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:02.155842 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1405/2829] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:54:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:54:04.054942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:04.056475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:04.056924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:04.078907 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] MFA Rules not processed for user `93b2cf587dc44209a186fc453ac5bdfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:54:04.080347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:04.170134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1425/2830] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:54:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:54:04.216296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:04.218396 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:04.235990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:54:04.310757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69babd61-20f9-4e8f-82ab-aeb3fdcec797 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:04.314163 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:04.314766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:04.315036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:04.315302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:04.317065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:04.334126 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:04.335595 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:04.347995 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1406/2831] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:05.729537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:05.731039 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:05.801891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:54:05.845718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b138b-ccf9-40bf-b39c-a11f12ecc0e2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:05.851350 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:05.851987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:05.852267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:05.852595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:05.854463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:05.866306 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:05.867894 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:05.877841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1426/2832] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:09.579997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:09.581713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:09.595191 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d86336-0ed6-4f0a-8598-9fc03bc0f29f service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:09.601178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:09.601787 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:09.602114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:09.602383 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:09.604073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:09.618916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:09.620563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:09.633060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1407/2833] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:14.631660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:14.644197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe27e-c5d4-4fae-b621-13d8166cd480 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:14.650000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:14.650594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:14.650875 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:14.651161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:14.653198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:14.665631 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:14.667198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c6e55-beea-4334-a0b2-94eea0f68cc8 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:14.678528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1427/2834] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:19.627086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=33 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:54:21.212940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:54:21.224577 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403f33-b995-4fd5-90d5-21c9a6d587eb service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:54:21.230113 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d44b8c931e047408a2d9d4baa490806', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:54:21.230740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:54:21.231026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:54:21.231298 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:54:21.233021 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:54:21.245246 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:54:21.246832 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:54:21.257809 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1408/2835] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:54:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:54:24.673094 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=21 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:54:31.252836 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:55:01.043664 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:55:01.044176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:55:01.044275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:55:01.062978 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] MFA Rules not processed for user `135b44145a1d4712b0ea9b3f00d1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:55:01.064158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:55:01.087154 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1428/2836] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:55:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:55:01.384587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:55:01.403812 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3add-8086-4b39-99a0-0465ffadd35b service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:55:01.407360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:55:01.407947 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:55:01.408219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:55:01.408485 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:55:01.410200 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:55:01.419985 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:55:01.421421 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:55:01.429097 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1409/2837] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:55:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:55:11.085102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=11 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:55:11.425626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=11 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:55:41.813651 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:55:41.815158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:55:41.836197 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ea60-c40e-45a5-be1d-c579f484468b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:55:41.840215 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:55:41.840809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:55:41.841087 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:55:41.841393 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:55:41.843083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:55:41.852347 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:55:41.854458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe369321-a65a-4dbd-84d5-9fa7187d8419 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:55:41.862368 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1429/2838] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:55:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:55:51.858776 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:55:52.070279 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:55:52.088992 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1ca19-4635-4698-bd3e-23d456417fb8 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:55:52.092994 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:55:52.093731 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:55:52.094073 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:55:52.094344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:55:52.096069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:55:52.127750 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:55:52.129176 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:55:52.149359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1410/2839] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:56:02.146502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=22 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:03.421295 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:03.421796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:03.421796 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:03.441646 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] MFA Rules not processed for user `135b44145a1d4712b0ea9b3f00d1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:56:03.442966 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:03.451390 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1430/2840] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:56:03.765694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:03.774343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543f1b-4ad2-4b72-933e-10c9d077a2c2 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:03.778281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:03.778883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:03.779162 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:03.779430 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:03.781152 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:03.790871 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:03.792290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:03.800589 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1411/2841] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:56:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:56:13.435718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:13.796808 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:18.052793 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:18.062942 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4542a5-1468-4984-8b08-4c5492005667 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:18.066493 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:18.067082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:18.067358 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:18.067627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:18.069335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:18.081365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:18.082842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:18.092182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1431/2842] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:56:20.857349 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:20.870197 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d93245-fc77-43d9-bfda-559e08260978 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:20.875657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:20.876255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:20.876530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:20.876827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:20.878702 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:20.890990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:20.892819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:20.905177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1412/2843] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:56:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:56:23.973571 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:23.975303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:23.992200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c2a7a-0632-4806-8cdc-1a6294bfa285 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:23.997523 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:23.998140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:23.998414 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:23.998683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] PATH_INFO: `/v3/users/93b2cf587dc44209a186fc453ac5bdfc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:23.999824 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b2cf587dc44209a186fc453ac5bdfc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:24.001754 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:24.048735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b2cf587dc44209a186fc453ac5bdfc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.049419 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b2cf587dc44209a186fc453ac5bdfc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.056494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b2cf587dc44209a186fc453ac5bdfc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.057167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b2cf587dc44209a186fc453ac5bdfc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.064399 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b2cf587dc44209a186fc453ac5bdfc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.081194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1432/2844] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:56:23 2022] DELETE /identity/v3/users/93b2cf587dc44209a186fc453ac5bdfc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:56:24.251928 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:24.262626 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330af7f-a0ac-4849-9f3a-3172b12650f5 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:24.319057 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:24.319675 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:24.319948 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:24.320217 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] PATH_INFO: `/v3/projects/df5069d6a3a34a08bb00105a91251cf2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:24.321345 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5069d6a3a34a08bb00105a91251cf2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:24.323231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:24.369909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5069d6a3a34a08bb00105a91251cf2', 'request_id': 'req-05045dda-3d36-4f02-95b0-46a5dbac1082'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.375972 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5069d6a3a34a08bb00105a91251cf2', 'request_id': 'req-05045dda-3d36-4f02-95b0-46a5dbac1082'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.381480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5069d6a3a34a08bb00105a91251cf2', 'request_id': 'req-05045dda-3d36-4f02-95b0-46a5dbac1082'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.393778 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] The token cache is being invalidate because project df5069d6a3a34a08bb00105a91251cf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:56:24.394062 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.394666 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.396132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1413/2845] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:56:24 2022] DELETE /identity/v3/projects/df5069d6a3a34a08bb00105a91251cf2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:56:24.400235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:24.402354 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:24.415521 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558dc51-5d73-4b1e-b1d8-fcbe5d786f8d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:24.471902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:24.472449 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:24.472728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:24.473000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] PATH_INFO: `/v3/users/f3324c5422254ead8a05136fce2856de` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:24.474157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3324c5422254ead8a05136fce2856de)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:24.476063 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:24.520980 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3324c5422254ead8a05136fce2856de'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.521650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3324c5422254ead8a05136fce2856de'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.529325 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3324c5422254ead8a05136fce2856de'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.529971 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3324c5422254ead8a05136fce2856de'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.533513 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:24.537261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3d57c3a0-ab7f-4236-8905-bdfc1b7e8f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3324c5422254ead8a05136fce2856de'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.555906 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1433/2846] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:56:24 2022] DELETE /identity/v3/users/f3324c5422254ead8a05136fce2856de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:56:24.724443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:56:24.725827 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:56:24.738574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05045dda-3d36-4f02-95b0-46a5dbac1082 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:56:24.796293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:56:24.796897 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:56:24.797196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:56:24.797466 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] PATH_INFO: `/v3/projects/9791973d73d74a3095f3e6346850f347` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:56:24.798556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9791973d73d74a3095f3e6346850f347)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:56:24.800598 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:56:24.820826 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:24.846555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9791973d73d74a3095f3e6346850f347', 'request_id': 'req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.852540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9791973d73d74a3095f3e6346850f347', 'request_id': 'req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.858028 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9791973d73d74a3095f3e6346850f347', 'request_id': 'req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.869969 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] The token cache is being invalidate because project 9791973d73d74a3095f3e6346850f347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:56:24.870789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.871376 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:56:24.872851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1414/2847] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:56:24 2022] DELETE /identity/v3/projects/9791973d73d74a3095f3e6346850f347 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:56:34.552770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:56:34.867016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:57:09.179232 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:57:09.179834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:57:09.179834 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:57:09.198698 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] MFA Rules not processed for user `135b44145a1d4712b0ea9b3f00d1bc30`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:57:09.199882 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:57:09.252249 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1434/2848] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:57:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:57:19.245815 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:57:50.142724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:57:50.144118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:57:50.212724 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc27b6-9b43-4ff8-a5f6-f7fc8b58239b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:57:50.217726 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:57:50.218324 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:57:50.218605 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:57:50.218880 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:57:50.220640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:57:50.232150 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:57:50.233764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:57:50.244047 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1415/2849] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:57:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:00.239885 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:20.961349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:20.963166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:21.036062 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881e1b1-53b6-4a98-a50c-a3a871f9d39f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.040457 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.041082 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.041386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.041685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] PATH_INFO: `/v3/users/135b44145a1d4712b0ea9b3f00d1bc30` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.042801 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135b44145a1d4712b0ea9b3f00d1bc30)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.044708 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.091944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135b44145a1d4712b0ea9b3f00d1bc30'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.092594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135b44145a1d4712b0ea9b3f00d1bc30'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.099762 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135b44145a1d4712b0ea9b3f00d1bc30'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.100392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135b44145a1d4712b0ea9b3f00d1bc30'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.107267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8f6a5e3a-4da1-47ea-9c51-f3e3579ace50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135b44145a1d4712b0ea9b3f00d1bc30'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.124707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1435/2850] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:20 2022] DELETE /identity/v3/users/135b44145a1d4712b0ea9b3f00d1bc30 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:21.297024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.310142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f93b73-ea6a-49a9-985a-e17164f3ef13 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.367422 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.367971 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.368245 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.368516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] PATH_INFO: `/v3/projects/462cfcb8aad746289ff3c2a01219da07` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.369640 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=462cfcb8aad746289ff3c2a01219da07)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.371518 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.417081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462cfcb8aad746289ff3c2a01219da07', 'request_id': 'req-3ee88617-23a3-4985-abab-8ade0766a1ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.424604 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462cfcb8aad746289ff3c2a01219da07', 'request_id': 'req-3ee88617-23a3-4985-abab-8ade0766a1ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.431486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462cfcb8aad746289ff3c2a01219da07', 'request_id': 'req-3ee88617-23a3-4985-abab-8ade0766a1ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.443654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] The token cache is being invalidate because project 462cfcb8aad746289ff3c2a01219da07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:21.443942 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.444556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3ee88617-23a3-4985-abab-8ade0766a1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:21.446053 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1416/2851] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:21 2022] DELETE /identity/v3/projects/462cfcb8aad746289ff3c2a01219da07 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:58:21.450883 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.451190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.451466 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.471413 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:21.472595 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:21.530330 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1436/2852] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:21.538851 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.539137 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.539415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.559240 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:21.560438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:21.568340 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1417/2853] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:21.572303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.581895 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833365a-3860-482a-8578-7fac29a3e3c2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.586421 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.587006 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.587287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.587555 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.588182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:58:21.588482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.590095 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.601997 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1437/2854] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:58:21.606069 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.615820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c16852-38e0-44cd-8767-0aacbfc65a71 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.620491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.621024 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.621334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.621608 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.622230 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.624100 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.650846 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1418/2855] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:21.655133 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.665400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0c6e3-acbb-4189-a349-52b88ed9de75 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.668863 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:21.725285 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.726037 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.726312 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.726591 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.727307 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.729790 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.755497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1438/2856] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:21.757234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.770273 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9919902-4c58-4737-ad0c-7c98712085a9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.775861 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.776498 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.776794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.777063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.777759 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.779619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.790502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1419/2857] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:58:21.794584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.804792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270854d-ce99-43a1-aed9-d197e1f73a0e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.809603 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.810287 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.810634 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.811030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.811866 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.813492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890beec-436f-4f0f-98fd-7c03b0b633f3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.824341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1439/2858] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:58:21.828329 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:21.839445 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940f844-3280-4caa-8b91-4481d86290d3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:21.844393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:21.845040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.845344 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.845621 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] PATH_INFO: `/v3/projects/0f8bd7c69e404ba5a59ec4b7386a7661/users/2fd85d5853b34cb2b8c8683a9dd0237a/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.860269 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8bd7c69e404ba5a59ec4b7386a7661, user_id=2fd85d5853b34cb2b8c8683a9dd0237a, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:21.862980 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:21.875265 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1420/2859] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:21 2022] PUT /identity/v3/projects/0f8bd7c69e404ba5a59ec4b7386a7661/users/2fd85d5853b34cb2b8c8683a9dd0237a/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:21.886333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:21.886617 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:21.886891 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:21.915460 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] MFA Rules not processed for user `2fd85d5853b34cb2b8c8683a9dd0237a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:21.916665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:21.978694 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1440/2860] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:21.983118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.011940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:22.045272 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6b9bb-3c2a-4acd-9969-e858dc055350 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.049925 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.050515 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.050790 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.051058 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.051680 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.053681 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.078892 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1421/2861] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:21 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:22.083774 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.094383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde31097-0c5f-4bf9-ad66-7bb5330f1492 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.151081 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.151609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.151889 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.152163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.152779 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.154979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.175588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1441/2862] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:22 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:22.179490 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.190004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0d76d-2375-4704-a0ec-679a0d4a92ca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.194569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.195153 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.195428 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.195696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.196321 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.197938 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.208805 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1422/2863] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:22.212737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.219007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:22.223500 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ddd0b-919e-40b0-b7be-c812f70a0052 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.229191 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.229858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.230134 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.230406 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] PATH_INFO: `/v3/projects/38b50abff31644e4a788d3624521cbb3/users/8d35ddba62d04143be1b2fc9bcfa3458/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.244844 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b50abff31644e4a788d3624521cbb3, user_id=8d35ddba62d04143be1b2fc9bcfa3458, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.247509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.260544 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1442/2864] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:22 2022] PUT /identity/v3/projects/38b50abff31644e4a788d3624521cbb3/users/8d35ddba62d04143be1b2fc9bcfa3458/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:22.264363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.274684 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139a901-11d2-4a2c-a388-340e4ff2b9b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.328407 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.329039 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.329290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.329561 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.330933 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.332960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d806f-3abc-45a3-b55c-c296640315f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.342566 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1423/2865] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:22.346702 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.356885 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25af20-a9e8-4037-94bf-15176632fc7c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.361482 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.362117 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.362392 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.362663 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] PATH_INFO: `/v3/projects/38b50abff31644e4a788d3624521cbb3/users/8d35ddba62d04143be1b2fc9bcfa3458/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.364574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b50abff31644e4a788d3624521cbb3, user_id=8d35ddba62d04143be1b2fc9bcfa3458, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.367274 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.379693 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1443/2866] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:22 2022] PUT /identity/v3/projects/38b50abff31644e4a788d3624521cbb3/users/8d35ddba62d04143be1b2fc9bcfa3458/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:22.389946 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.390257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.390535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.421966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] MFA Rules not processed for user `8d35ddba62d04143be1b2fc9bcfa3458`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:22.423130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:22.490820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1424/2867] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:22.501913 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:22.504169 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:22.562000 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea86505-357e-45d5-a385-994c445918d2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:22.566282 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:22.566874 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:22.567152 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:22.567422 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:22.576103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:22.577566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87825ce5-ac0c-42d1-87c6-f1aaca65315c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:22.586728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1444/2868] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:22.991040 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:23.014819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:23.052776 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd452b3-cd36-4e3e-ad9e-d8954a2b5c58 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:23.057271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:23.057889 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:23.058167 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:23.058438 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] PATH_INFO: `/v3/users/2fd85d5853b34cb2b8c8683a9dd0237a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:23.059541 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd85d5853b34cb2b8c8683a9dd0237a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:23.061486 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:23.104042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd85d5853b34cb2b8c8683a9dd0237a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.110751 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd85d5853b34cb2b8c8683a9dd0237a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.118203 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd85d5853b34cb2b8c8683a9dd0237a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.118837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd85d5853b34cb2b8c8683a9dd0237a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.130378 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd85d5853b34cb2b8c8683a9dd0237a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.136129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1425/2869] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:22 2022] DELETE /identity/v3/users/2fd85d5853b34cb2b8c8683a9dd0237a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:23.140641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:23.141130 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:23.141223 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:23.160741 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:23.161975 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:23.219321 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1445/2870] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:23.229877 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:23.231288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:23.287863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e88c-f218-47a2-9c06-9fdbadb64ac3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:23.291420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:23.292006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:23.292280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:23.292547 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:23.301432 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:23.302847 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:23.311957 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1426/2871] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:23.571495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:23.573009 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:23.585916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca4aa1-68e1-4472-9f4d-d9d1479e43a9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:23.590569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:23.591196 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:23.591468 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:23.591735 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] PATH_INFO: `/v3/projects/0f8bd7c69e404ba5a59ec4b7386a7661` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:23.599543 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8bd7c69e404ba5a59ec4b7386a7661)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:23.601510 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:23.647604 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8bd7c69e404ba5a59ec4b7386a7661', 'request_id': 'req-41357cb6-535f-485a-a678-1581e32937d6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.654588 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8bd7c69e404ba5a59ec4b7386a7661', 'request_id': 'req-41357cb6-535f-485a-a678-1581e32937d6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.660416 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8bd7c69e404ba5a59ec4b7386a7661', 'request_id': 'req-41357cb6-535f-485a-a678-1581e32937d6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.661498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:23.673769 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] The token cache is being invalidate because project 0f8bd7c69e404ba5a59ec4b7386a7661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:23.674054 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.674722 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.676270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1446/2872] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:23 2022] DELETE /identity/v3/projects/0f8bd7c69e404ba5a59ec4b7386a7661 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:23.680257 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:23.690769 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9a1e-e578-48c9-b15c-33abe0f7ef26 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:23.732060 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:23.749352 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:23.749914 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:23.750190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:23.750459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] PATH_INFO: `/v3/users/8d35ddba62d04143be1b2fc9bcfa3458` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:23.751619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d35ddba62d04143be1b2fc9bcfa3458)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:23.754099 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:23.801380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d35ddba62d04143be1b2fc9bcfa3458'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.809372 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d35ddba62d04143be1b2fc9bcfa3458'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.818509 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d35ddba62d04143be1b2fc9bcfa3458'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.819127 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d35ddba62d04143be1b2fc9bcfa3458'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.831384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-40442dd6-6df9-497f-9b76-c5170401df92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d35ddba62d04143be1b2fc9bcfa3458'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:23.837094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1427/2873] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:23 2022] DELETE /identity/v3/users/8d35ddba62d04143be1b2fc9bcfa3458 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:24.104537 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.106171 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:24.119725 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41357cb6-535f-485a-a678-1581e32937d6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.175959 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.176600 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.176858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.177164 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] PATH_INFO: `/v3/projects/38b50abff31644e4a788d3624521cbb3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.178309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b50abff31644e4a788d3624521cbb3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.180212 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.225365 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b50abff31644e4a788d3624521cbb3', 'request_id': 'req-a65fc8b6-a859-471d-8e3d-61bd66d04c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:24.232355 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b50abff31644e4a788d3624521cbb3', 'request_id': 'req-a65fc8b6-a859-471d-8e3d-61bd66d04c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:24.238299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b50abff31644e4a788d3624521cbb3', 'request_id': 'req-a65fc8b6-a859-471d-8e3d-61bd66d04c50'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:24.248878 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] The token cache is being invalidate because project 38b50abff31644e4a788d3624521cbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:24.249195 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:24.249819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a65fc8b6-a859-471d-8e3d-61bd66d04c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:24.251263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1447/2874] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:24 2022] DELETE /identity/v3/projects/38b50abff31644e4a788d3624521cbb3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:24.256720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.257001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.257307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.276231 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:24.277502 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:24.336118 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1428/2875] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:24.344008 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.344291 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.344566 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.364833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:24.366055 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:24.374573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1448/2876] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:24.378744 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.388357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1589a5-6e22-45ca-9c03-8660f9391951 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.392940 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.393574 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.393855 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.394129 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.394773 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:58:24.395102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.396693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.408895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1429/2877] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:58:24.413308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.423045 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c32a0-b5a5-4ef1-90de-f422080829c1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.428526 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.429141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.429440 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.429715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.430352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.432275 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.459791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1449/2878] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:24.463929 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.473794 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246b957-1989-4f30-83d5-dcd4f6488261 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.520687 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:24.533819 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.534363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.534646 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.534920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.535571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.537531 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.558178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1430/2879] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:24.562535 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.573398 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2bb5f-318c-43d9-8c91-c62d7084a9c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.578382 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.579161 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.579451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.579806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.581018 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.582684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.592804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1450/2880] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:24.596959 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.607114 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a193-871b-4ae9-a1d3-a625ef2620ba admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.611682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.612267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.612544 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.612813 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.613487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.615098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defc468-9fc9-4408-8497-54906b6823e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.625402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1431/2881] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:58:24.629806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.639627 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb7dd4-382d-4c52-b76a-73577bbe8738 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.644848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.645522 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.645798 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.646073 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] PATH_INFO: `/v3/projects/337ccadc3f7c4c11a2757c401a860e50/users/fc75de0c4da6447b870ab7f7f2cd812a/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.660922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] RBAC: Authorizing `identity:create_grant(project_id=337ccadc3f7c4c11a2757c401a860e50, user_id=fc75de0c4da6447b870ab7f7f2cd812a, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.663594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.676559 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1451/2882] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:24 2022] PUT /identity/v3/projects/337ccadc3f7c4c11a2757c401a860e50/users/fc75de0c4da6447b870ab7f7f2cd812a/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:58:24.687745 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.688222 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.688311 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.715555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] MFA Rules not processed for user `fc75de0c4da6447b870ab7f7f2cd812a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:24.716754 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:24.786133 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1432/2883] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:24.790509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.811352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:24.854653 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf687-7e27-4c6c-a876-813390f3586e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.859270 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.859868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.860146 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.860417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.861043 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.863015 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.889041 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1452/2884] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:24.894111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:24.905232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83113681-56af-4bf6-8d7b-f41c590d6323 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:24.960642 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:24.961277 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:24.961553 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:24.961822 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:24.962440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:24.964328 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:24.985958 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1433/2885] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:24 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:24.990090 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.000166 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39647-803c-488e-9f1a-c1059d1f8068 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.004849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.005477 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.005756 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.006030 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.006674 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.008265 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.019167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1453/2886] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:25.023286 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.034165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68811b80-d324-4393-b516-24ff59cdf30c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.038895 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.039482 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.039763 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.040165 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] PATH_INFO: `/v3/projects/f6abeee861494c629bdb188c5ae28b86/users/9050ab3f60914f549f86a9eae58ce7ab/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.056360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6abeee861494c629bdb188c5ae28b86, user_id=9050ab3f60914f549f86a9eae58ce7ab, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.059339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.074734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1434/2887] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:25 2022] PUT /identity/v3/projects/f6abeee861494c629bdb188c5ae28b86/users/9050ab3f60914f549f86a9eae58ce7ab/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:25.078791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.089349 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac61c6-e694-41fc-9813-f47a9ab455e7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.144580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.145209 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.145495 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.145770 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.146405 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.147987 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ff681-5e5f-4dcc-a384-14fe762cdef7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.157789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1454/2888] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:25.161793 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.171875 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286065b-fc11-4a44-a64f-c733ddc84d04 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.176411 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.177048 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.177359 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.177632 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] PATH_INFO: `/v3/projects/f6abeee861494c629bdb188c5ae28b86/users/9050ab3f60914f549f86a9eae58ce7ab/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.179592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6abeee861494c629bdb188c5ae28b86, user_id=9050ab3f60914f549f86a9eae58ce7ab, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.182261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.192719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1435/2889] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:25 2022] PUT /identity/v3/projects/f6abeee861494c629bdb188c5ae28b86/users/9050ab3f60914f549f86a9eae58ce7ab/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:25.202948 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.203404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.203492 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.231198 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] MFA Rules not processed for user `9050ab3f60914f549f86a9eae58ce7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:25.232401 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:25.297806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1455/2890] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:25.308263 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.310031 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:25.324079 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:25.367271 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c1a27-bb68-4cfe-9b39-63f3cc35e891 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.370887 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.371480 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.371756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.372026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.381255 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.382677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.392151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1436/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:25.493599 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:25.503791 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e64f7-3914-47a1-89e9-c6aa2b0fa53e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:25.507353 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:25.507944 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:25.508218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:25.508485 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:25.520247 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:25.566278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:25.567650 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd8fd2-0c84-4e9a-a779-4ee3740c60ef service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:25.576463 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1456/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:28.145023 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:28.204086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04f7d6-3d22-4102-8595-6b7437d77c20 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:28.209141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:28.209764 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:28.210036 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:28.210304 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] PATH_INFO: `/v3/users/fc75de0c4da6447b870ab7f7f2cd812a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:28.211395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc75de0c4da6447b870ab7f7f2cd812a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:28.213319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:28.256987 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc75de0c4da6447b870ab7f7f2cd812a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.263600 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc75de0c4da6447b870ab7f7f2cd812a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.271042 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc75de0c4da6447b870ab7f7f2cd812a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.271654 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc75de0c4da6447b870ab7f7f2cd812a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.283234 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc75de0c4da6447b870ab7f7f2cd812a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.288923 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1437/2893] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:28 2022] DELETE /identity/v3/users/fc75de0c4da6447b870ab7f7f2cd812a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:28.293447 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:28.293896 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:28.294007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:28.313141 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:28.314348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:28.372007 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1457/2894] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:28.382095 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:28.383501 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:28.420339 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:28.444115 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ad978-ecd5-4e4a-89cb-7c4378939c17 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:28.447704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:28.448302 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:28.448580 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:28.448844 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:28.457583 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:28.458997 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:28.468149 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1438/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:28.730630 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:28.732252 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:28.746026 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c51643-3bc0-46df-a438-faec7343d221 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:28.750360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:28.750947 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:28.751226 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:28.751498 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] PATH_INFO: `/v3/projects/337ccadc3f7c4c11a2757c401a860e50` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:28.752641 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=337ccadc3f7c4c11a2757c401a860e50)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:28.755176 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:28.812348 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337ccadc3f7c4c11a2757c401a860e50', 'request_id': 'req-9cce7155-42d8-401d-b6e6-e49673cdd274'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.820202 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337ccadc3f7c4c11a2757c401a860e50', 'request_id': 'req-9cce7155-42d8-401d-b6e6-e49673cdd274'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.825890 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337ccadc3f7c4c11a2757c401a860e50', 'request_id': 'req-9cce7155-42d8-401d-b6e6-e49673cdd274'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.836728 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] The token cache is being invalidate because project 337ccadc3f7c4c11a2757c401a860e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:28.837012 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.837665 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.839143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1458/2896] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:28 2022] DELETE /identity/v3/projects/337ccadc3f7c4c11a2757c401a860e50 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:28.843064 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:28.853081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a982-c3f4-4e2c-806f-57029a56ecb5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:28.907107 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:28.907647 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:28.907919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:28.908188 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] PATH_INFO: `/v3/users/9050ab3f60914f549f86a9eae58ce7ab` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:28.909314 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9050ab3f60914f549f86a9eae58ce7ab)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:28.911691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:28.955219 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9050ab3f60914f549f86a9eae58ce7ab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.962001 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9050ab3f60914f549f86a9eae58ce7ab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.968978 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9050ab3f60914f549f86a9eae58ce7ab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.969606 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9050ab3f60914f549f86a9eae58ce7ab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.981555 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-60732f1d-04a8-4c9f-bbe8-ae829467e1c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9050ab3f60914f549f86a9eae58ce7ab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:28.987242 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1439/2897] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:28 2022] DELETE /identity/v3/users/9050ab3f60914f549f86a9eae58ce7ab => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:29.283552 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.285261 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:29.299624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce7155-42d8-401d-b6e6-e49673cdd274 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.335622 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:29.359298 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.359924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.360201 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.360471 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] PATH_INFO: `/v3/projects/f6abeee861494c629bdb188c5ae28b86` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.361596 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6abeee861494c629bdb188c5ae28b86)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.364036 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.409638 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6abeee861494c629bdb188c5ae28b86', 'request_id': 'req-0f69d247-9801-4c06-917c-87fb128c79ac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:29.417108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6abeee861494c629bdb188c5ae28b86', 'request_id': 'req-0f69d247-9801-4c06-917c-87fb128c79ac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:29.422783 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6abeee861494c629bdb188c5ae28b86', 'request_id': 'req-0f69d247-9801-4c06-917c-87fb128c79ac'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:29.434240 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] The token cache is being invalidate because project f6abeee861494c629bdb188c5ae28b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:29.434528 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:29.435163 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0f69d247-9801-4c06-917c-87fb128c79ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:29.436644 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1459/2898] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:29 2022] DELETE /identity/v3/projects/f6abeee861494c629bdb188c5ae28b86 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:29.442084 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.442534 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.442619 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.461830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:29.462956 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:29.509415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:29.522132 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1440/2899] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:29.531430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.531715 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.531992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.552341 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:29.553578 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:29.561383 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1460/2900] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:29.565380 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.575496 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca255a6-2eae-4fdd-8d6a-1035e3b0cd52 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.580077 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.580677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.580953 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.581260 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.581904 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:58:29.582205 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.583830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.595086 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1441/2901] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:58:29.599749 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.608775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdccdd7-836a-44a8-8109-e2c7b8d93052 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.613949 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.614529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.614806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.615075 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.615684 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.617586 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.644587 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1461/2902] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:29.648765 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.658841 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36280e5-a311-4497-83f9-5cab87c642e5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.713288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.713823 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.714236 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.714394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.715004 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.717006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.739384 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1442/2903] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:29.743431 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.753789 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5341fa-f2fa-40a4-bbd6-9acc02491c4c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.760692 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.761338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.761623 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.761900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.762558 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.764718 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.776318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1462/2904] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:29.780679 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.791330 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785c8c5-889e-47a3-a068-279827c6b73a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.795982 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.796576 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.796856 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.797142 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.797807 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.799415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399c85c-d896-402f-ae09-495b42480713 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.811507 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1443/2905] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:29.815575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:29.825491 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2162f-3c47-4606-bf33-250cfd00f6be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:29.830103 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:29.830736 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.831014 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.831289 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] PATH_INFO: `/v3/projects/46bcbf299e8142f7894f1ae851fa1f15/users/6bb1b2aaa2124fe9952a1d81fba066ff/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.848224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46bcbf299e8142f7894f1ae851fa1f15, user_id=6bb1b2aaa2124fe9952a1d81fba066ff, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:29.850938 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:29.864483 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1463/2906] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:29 2022] PUT /identity/v3/projects/46bcbf299e8142f7894f1ae851fa1f15/users/6bb1b2aaa2124fe9952a1d81fba066ff/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:29.874720 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:29.875009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:29.875307 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:29.902288 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] MFA Rules not processed for user `6bb1b2aaa2124fe9952a1d81fba066ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:29.903484 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:29.967682 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1444/2907] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:29.972097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.033001 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505fa4c-872f-4de8-85f5-1887543bb1e1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.038140 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.038726 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.039003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.039273 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.039897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.041808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.050823 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:30.070051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1464/2908] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:29 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:58:30.074281 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.085718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cc008-5e9f-4066-bef0-484a1754ca89 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.135261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:30.146365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.147016 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.147297 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.147683 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.148317 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.150262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.171226 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1445/2909] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:30 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:30.175222 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.185400 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0007fb-9879-47fe-ab09-91b7f4ba94e7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.190378 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.190979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.191278 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.191549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.192189 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.193828 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.203579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1465/2910] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 07:58:30.207741 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.217582 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983b879-5c1f-46e2-880a-e526485751ae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.222220 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.223332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.223611 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.223883 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.224512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.226124 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b780b50-9b30-46c4-9ca8-86aa5080b258 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.235713 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1446/2911] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:30.239720 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.249549 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedaa38-8abd-4bc1-b179-9c80c5728bd9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.254841 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.255511 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.255805 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.256077 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] PATH_INFO: `/v3/projects/213da516b92145a8bf24a2e556e1fa2e/users/1395d5326149491ebc5d76203d61abb9/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.271100 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213da516b92145a8bf24a2e556e1fa2e, user_id=1395d5326149491ebc5d76203d61abb9, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.273777 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.286514 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1466/2912] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:30 2022] PUT /identity/v3/projects/213da516b92145a8bf24a2e556e1fa2e/users/1395d5326149491ebc5d76203d61abb9/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:30.296691 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.296973 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.297283 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.325402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] MFA Rules not processed for user `1395d5326149491ebc5d76203d61abb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:30.326571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:30.389397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1447/2913] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:30.393924 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.458820 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c6a9-0701-413a-afd5-4f3bd604af93 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.463619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.464243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.464489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.464764 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.465428 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.467370 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.497478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1467/2914] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:30 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:30.498658 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.510177 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f61fa1-c3c6-489d-850e-28cdb3033efe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.566747 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.567347 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.567628 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.567896 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.568514 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.570450 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.591734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1448/2915] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:30 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:58:30.595837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.606423 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295b793-01d6-4ef5-8842-18bf27ccf29d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.611352 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.611964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.612243 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.612515 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.613182 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.614775 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.625439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1468/2916] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:30.629950 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.639624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ac4a-261a-4297-b8f7-ebc753949423 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.644102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.645332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.645610 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.645882 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] PATH_INFO: `/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e/users/6f16421e33ac493098c62dc19ba2d69e/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.660209 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051bfdd9d31b490ea2e72f9d4b4fc57e, user_id=6f16421e33ac493098c62dc19ba2d69e, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.662881 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.675214 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1449/2917] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:30 2022] PUT /identity/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e/users/6f16421e33ac493098c62dc19ba2d69e/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:30.679532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.689397 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032e1f2-9845-45e9-a4f8-31112f02e369 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.744567 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.745097 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.745404 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.745679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.746318 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.747945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493037d-88f7-4397-bd08-3f2f3ff37283 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.758808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1469/2918] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:30.762919 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.773363 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bc8ae-0caf-49e0-8c12-0c29cb728e19 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.777976 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.778614 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.778893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.779164 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] PATH_INFO: `/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e/users/6f16421e33ac493098c62dc19ba2d69e/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.781459 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=051bfdd9d31b490ea2e72f9d4b4fc57e, user_id=6f16421e33ac493098c62dc19ba2d69e, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.784130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:30.794305 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1450/2919] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:30 2022] PUT /identity/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e/users/6f16421e33ac493098c62dc19ba2d69e/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:58:30.805849 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.806138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.806412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.835116 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] MFA Rules not processed for user `6f16421e33ac493098c62dc19ba2d69e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:30.836339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:30.847813 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:30.904848 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1470/2920] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:30.915937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:30.917660 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:30.932301 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:30.975983 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae63d5-3c93-4635-874c-19b76e97c9f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:30.979698 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:30.980290 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:30.980563 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:30.980830 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:30.990111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:30.991526 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:31.000762 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1451/2921] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:31.045060 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:31.046580 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:31.059002 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320b8ad-0264-4ecb-ba05-bb8bfa9a985b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:31.062569 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:31.063159 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:31.063436 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:31.063707 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:31.121190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:31.122974 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff54485-6cdc-454c-b5db-9c931583e9ad service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:31.131424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1471/2922] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:33.988373 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:34.050332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed1e8a-1353-4ddc-92a9-2ba8c1ac69e3 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:34.054843 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:34.055458 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:34.055736 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:34.056007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] PATH_INFO: `/v3/users/6bb1b2aaa2124fe9952a1d81fba066ff` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:34.057102 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb1b2aaa2124fe9952a1d81fba066ff)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:34.059038 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:34.103667 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb1b2aaa2124fe9952a1d81fba066ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.110394 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb1b2aaa2124fe9952a1d81fba066ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.117767 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb1b2aaa2124fe9952a1d81fba066ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.118385 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb1b2aaa2124fe9952a1d81fba066ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.130109 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb1b2aaa2124fe9952a1d81fba066ff'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.136403 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1452/2923] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:33 2022] DELETE /identity/v3/users/6bb1b2aaa2124fe9952a1d81fba066ff => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:34.140886 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:34.141364 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:34.141592 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:34.161945 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:34.163136 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:34.219360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1472/2924] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:34.229334 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:34.230719 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:34.266094 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:34.289696 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240defe5-fcc0-4635-be1a-cd366ad37eec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:34.293262 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:34.293858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:34.294130 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:34.294397 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:34.303120 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:34.305141 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:34.314587 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1453/2925] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:34.585671 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:34.587221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:34.606771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b2e5e-16bc-4f3c-9b14-52246ed2ae69 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:34.612079 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:34.612865 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:34.613254 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:34.613679 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] PATH_INFO: `/v3/projects/46bcbf299e8142f7894f1ae851fa1f15` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:34.614922 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46bcbf299e8142f7894f1ae851fa1f15)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:34.617556 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:34.660231 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:34.664309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46bcbf299e8142f7894f1ae851fa1f15', 'request_id': 'req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.671635 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46bcbf299e8142f7894f1ae851fa1f15', 'request_id': 'req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.677093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46bcbf299e8142f7894f1ae851fa1f15', 'request_id': 'req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.688305 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] The token cache is being invalidate because project 46bcbf299e8142f7894f1ae851fa1f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:34.688593 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.689244 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.690713 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1473/2926] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:34 2022] DELETE /identity/v3/projects/46bcbf299e8142f7894f1ae851fa1f15 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:58:34.695280 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:34.704886 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3b4b2-a618-4278-a2d3-47db42ba675e service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:34.760185 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:34.760766 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:34.761052 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:34.761356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] PATH_INFO: `/v3/users/1395d5326149491ebc5d76203d61abb9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:34.762483 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1395d5326149491ebc5d76203d61abb9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:34.764424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:34.810716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1395d5326149491ebc5d76203d61abb9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.817685 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1395d5326149491ebc5d76203d61abb9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.824742 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1395d5326149491ebc5d76203d61abb9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.825395 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1395d5326149491ebc5d76203d61abb9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.836965 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1395d5326149491ebc5d76203d61abb9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:34.842785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1454/2927] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:34 2022] DELETE /identity/v3/users/1395d5326149491ebc5d76203d61abb9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:35.105855 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:35.107442 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:35.125574 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c93fd5-d0b9-4da6-9ac3-dcded7b111c1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:35.182557 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:35.183190 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.183443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.183737 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] PATH_INFO: `/v3/projects/213da516b92145a8bf24a2e556e1fa2e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.184837 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213da516b92145a8bf24a2e556e1fa2e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:35.186759 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:35.232430 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213da516b92145a8bf24a2e556e1fa2e', 'request_id': 'req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.239260 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213da516b92145a8bf24a2e556e1fa2e', 'request_id': 'req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.244687 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213da516b92145a8bf24a2e556e1fa2e', 'request_id': 'req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.255228 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] The token cache is being invalidate because project 213da516b92145a8bf24a2e556e1fa2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:35.255509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.256143 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.257685 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1474/2928] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:35 2022] DELETE /identity/v3/projects/213da516b92145a8bf24a2e556e1fa2e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:35.261733 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:35.263159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:35.276106 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c8c81-9978-4979-b91b-88c0f7321fdf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:35.324740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:35.332415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:35.333319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.333800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.334121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] PATH_INFO: `/v3/users/6f16421e33ac493098c62dc19ba2d69e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.335912 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f16421e33ac493098c62dc19ba2d69e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:35.337865 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:35.382443 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f16421e33ac493098c62dc19ba2d69e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.390267 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f16421e33ac493098c62dc19ba2d69e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.398160 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f16421e33ac493098c62dc19ba2d69e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.398784 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f16421e33ac493098c62dc19ba2d69e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.410718 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b846e003-4d29-48cd-986b-4809aec66747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f16421e33ac493098c62dc19ba2d69e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.417247 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1455/2929] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:35 2022] DELETE /identity/v3/users/6f16421e33ac493098c62dc19ba2d69e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:35.686158 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:35.687654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:35.700670 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb72da3-85f7-4c39-a20e-a72b2ca49ec0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:35.762144 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:35.762780 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.763059 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.763333 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] PATH_INFO: `/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.764458 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=051bfdd9d31b490ea2e72f9d4b4fc57e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:35.766389 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:35.768682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:35.813004 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '051bfdd9d31b490ea2e72f9d4b4fc57e', 'request_id': 'req-f55913de-7e21-4068-bf63-d734b818ee06'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.819897 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '051bfdd9d31b490ea2e72f9d4b4fc57e', 'request_id': 'req-f55913de-7e21-4068-bf63-d734b818ee06'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.825360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '051bfdd9d31b490ea2e72f9d4b4fc57e', 'request_id': 'req-f55913de-7e21-4068-bf63-d734b818ee06'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.836053 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] The token cache is being invalidate because project 051bfdd9d31b490ea2e72f9d4b4fc57e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:35.836338 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.836964 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f55913de-7e21-4068-bf63-d734b818ee06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:35.838475 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1475/2930] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:35 2022] DELETE /identity/v3/projects/051bfdd9d31b490ea2e72f9d4b4fc57e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:35.844357 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.844643 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.844920 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.864211 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:35.865512 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:35.923244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1456/2931] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:35.932023 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.932308 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.932584 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.952448 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:35.953683 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:35.961602 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1476/2932] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:35.965694 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:35.975168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0f1e5-222d-4559-89a3-632357d7dbc1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:35.980012 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:35.980620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:35.980916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:35.981206 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:35.981853 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:58:35.982161 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:35.983772 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:35.996111 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1457/2933] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:58:36.000490 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.010320 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a68a53-5729-476f-b71b-228888a9f29c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.014979 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.015563 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.015839 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.016109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.016742 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.018675 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.044433 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1477/2934] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:35 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:36.048491 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.058837 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9d5c-b165-40a9-9ebb-136e41b0487c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.112668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.113293 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.113637 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.113905 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.114624 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.116966 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.125676 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:36.138909 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1458/2935] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:36 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 07:58:36.143102 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.153337 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fd6f-9689-4424-84e2-8970f77effd2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.158177 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.158795 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.159074 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.159343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.159995 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.161616 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.171211 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1478/2936] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:36.174761 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.185009 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da6e1c-b296-40af-880d-21db3906a0b8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.189599 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.190190 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.190470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.190746 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.191382 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.192962 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee8407-d247-4bb5-927e-e8ae102b5400 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.203752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1459/2937] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:36.207598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.217313 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43a7c-5396-4032-8433-d1008b55b448 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.221902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.222529 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.222808 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.223080 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] PATH_INFO: `/v3/projects/21831ae4f77846c3a7ad366502f75f2c/users/bf9a3fccd4a64300af8b0774a4ca2225/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.238132 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=21831ae4f77846c3a7ad366502f75f2c, user_id=bf9a3fccd4a64300af8b0774a4ca2225, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.240842 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.254326 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1479/2938] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:36 2022] PUT /identity/v3/projects/21831ae4f77846c3a7ad366502f75f2c/users/bf9a3fccd4a64300af8b0774a4ca2225/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:36.266470 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.266934 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.267022 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.294487 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] MFA Rules not processed for user `bf9a3fccd4a64300af8b0774a4ca2225`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:36.295693 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:36.361800 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1460/2939] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:36.367210 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.430530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40292db1-d9fd-465a-aec4-103fc3084ffa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.435598 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.436194 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.436473 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.436746 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.437424 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.439339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.467051 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1480/2940] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:36 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:36.469516 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.480729 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d133a-23f1-4406-a66f-03c6f2850e57 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.537735 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.538867 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.539159 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.539415 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.540041 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.541991 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.562990 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1461/2941] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:36 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:36.566927 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.579309 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54383c9-95fb-41bf-83ee-9713f8b01c75 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.583920 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.584509 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.584785 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.585594 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.586224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.587818 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.598489 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1481/2942] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:36.603184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.613356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f764ea-1c8b-4107-ba5b-97f45532dd3e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.618752 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.619343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.619620 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.619891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] PATH_INFO: `/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4/users/c7e6b8c53137465d801df367546cca18/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.635571 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ad28f43fff4cbd8a7924909d8daab4, user_id=c7e6b8c53137465d801df367546cca18, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.638232 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.650924 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1462/2943] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:36 2022] PUT /identity/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4/users/c7e6b8c53137465d801df367546cca18/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:36.655109 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.660235 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:36.666962 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659df1fc-91cf-4123-8824-edf7c3196fc5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.723200 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.723792 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.724068 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.724339 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.724968 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.727138 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfb54d-3cf4-4ec3-9577-3b95db33b68b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.736505 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1482/2944] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:36.740689 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.750709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a21a5-1ae2-440d-be8b-ef17936b857a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:36.755494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:36.756147 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.756424 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.756695 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] PATH_INFO: `/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4/users/c7e6b8c53137465d801df367546cca18/roles/9a0636d853ba4c5d9cdae17e523d036d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.763050 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ad28f43fff4cbd8a7924909d8daab4, user_id=c7e6b8c53137465d801df367546cca18, role_id=9a0636d853ba4c5d9cdae17e523d036d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:36.766536 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:36.780261 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1463/2945] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:36 2022] PUT /identity/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4/users/c7e6b8c53137465d801df367546cca18/roles/9a0636d853ba4c5d9cdae17e523d036d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:36.852706 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:36.852992 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:36.853303 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:36.880619 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] MFA Rules not processed for user `c7e6b8c53137465d801df367546cca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:36.881809 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:36.948530 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1483/2946] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:36.959390 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:36.961703 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:36.987538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:37.029017 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c6d-a3b5-441a-9e35-09d1d0427bb4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:37.032891 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:37.033538 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:37.033814 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:37.034081 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:37.043108 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:37.044532 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbc8d2-2ec7-4580-812f-32c1b027e0fd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:37.053704 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1464/2947] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:37.415263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:37.476744 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19ef38-ce05-48fa-ba7d-a82a5191e5f1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:37.481286 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:37.481902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:37.482179 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:37.482451 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] PATH_INFO: `/v3/users/bf9a3fccd4a64300af8b0774a4ca2225` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:37.483542 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9a3fccd4a64300af8b0774a4ca2225)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:37.485486 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:37.513579 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:37.533751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9a3fccd4a64300af8b0774a4ca2225'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:37.534427 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9a3fccd4a64300af8b0774a4ca2225'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:37.541363 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9a3fccd4a64300af8b0774a4ca2225'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:37.541991 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9a3fccd4a64300af8b0774a4ca2225'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:37.549234 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9a3fccd4a64300af8b0774a4ca2225'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:37.566916 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1484/2948] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:37 2022] DELETE /identity/v3/users/bf9a3fccd4a64300af8b0774a4ca2225 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:37.572088 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:37.572562 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:37.572677 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:37.590709 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:37.591908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:37.647734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1465/2949] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:37.657506 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:37.658868 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:37.714472 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04742a-1c3c-4346-bb5f-ad016cea2ebf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:37.718093 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:37.718682 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:37.718955 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:37.719221 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:37.728647 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:37.730083 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:37.739267 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1485/2950] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 07:58:38.005420 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.006866 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:38.021556 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0d297-a918-4c76-acf1-b599017d6b68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.026960 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.027607 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.027916 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.028195 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] PATH_INFO: `/v3/projects/21831ae4f77846c3a7ad366502f75f2c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.029332 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=21831ae4f77846c3a7ad366502f75f2c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.031540 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.077423 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21831ae4f77846c3a7ad366502f75f2c', 'request_id': 'req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.083457 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21831ae4f77846c3a7ad366502f75f2c', 'request_id': 'req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.089545 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21831ae4f77846c3a7ad366502f75f2c', 'request_id': 'req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.102244 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] The token cache is being invalidate because project 21831ae4f77846c3a7ad366502f75f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:38.102535 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.103168 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.104625 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1466/2951] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:38 2022] DELETE /identity/v3/projects/21831ae4f77846c3a7ad366502f75f2c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:38.108609 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.118263 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bf924-92ca-4bfb-836c-fa522add5525 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.174218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.174835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.175115 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.175386 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] PATH_INFO: `/v3/users/c7e6b8c53137465d801df367546cca18` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.176497 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e6b8c53137465d801df367546cca18)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.178441 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.223269 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e6b8c53137465d801df367546cca18'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.223835 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e6b8c53137465d801df367546cca18'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.225028 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:38.233439 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e6b8c53137465d801df367546cca18'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.234157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e6b8c53137465d801df367546cca18'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.242206 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1f535f0a-adb9-464c-9bf6-6e6fe9b0b598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e6b8c53137465d801df367546cca18'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.259151 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1486/2952] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:38 2022] DELETE /identity/v3/users/c7e6b8c53137465d801df367546cca18 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:38.526936 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.528757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:38.543592 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297d2c1-8aa8-4093-babe-6c2f2f1756ac admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.581612 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:38.600791 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.601453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.601732 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.602007 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] PATH_INFO: `/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.603121 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ad28f43fff4cbd8a7924909d8daab4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.605018 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.649026 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ad28f43fff4cbd8a7924909d8daab4', 'request_id': 'req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.655319 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ad28f43fff4cbd8a7924909d8daab4', 'request_id': 'req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.660663 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ad28f43fff4cbd8a7924909d8daab4', 'request_id': 'req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.672500 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] The token cache is being invalidate because project b8ad28f43fff4cbd8a7924909d8daab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:38.672789 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.673452 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-409c88e3-3c9c-43f0-b328-cde8b6d3a66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:38.674910 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1467/2953] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:38 2022] DELETE /identity/v3/projects/b8ad28f43fff4cbd8a7924909d8daab4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:38.679819 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.680335 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.680425 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.700145 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:38.701373 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:38.759624 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1487/2954] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:38.767296 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.768027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.768027 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.787820 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:38.789006 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:38.796757 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1468/2955] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:38.800923 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.811167 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7a07-f933-46f7-ad97-bbf4402eafbd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.815833 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.816420 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.816701 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.816976 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.817660 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 07:58:38.817969 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.820213 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.831507 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1488/2956] 10.222.0.46 () {58 vars in 1153 bytes} [Thu Sep 1 07:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 07:58:38.835557 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.844710 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1a97d-1bc9-4ff0-bdee-564c180ec55d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.849591 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.850196 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.850477 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.850756 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.851413 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.853343 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.879831 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1469/2957] 10.222.0.46 () {60 vars in 1153 bytes} [Thu Sep 1 07:58:38 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:38.884478 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.894343 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761db42-a3ee-4d4e-8adc-18ea6fc69a4c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.950417 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.951016 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.951299 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.951575 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.952207 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.954157 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:38.974494 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1489/2958] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Sep 1 07:58:38 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 07:58:38.978572 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:38.988734 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d510a-b401-43dc-8d67-3e04c34ee372 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:38.993970 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:38.994560 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:38.994833 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:38.995098 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:38.995748 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:38.997356 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.007552 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1470/2959] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:39.011322 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.021276 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62304d5-0ca1-4522-90e1-c161d3e531a0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.026035 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.026658 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.026939 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.027216 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.027852 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.029487 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a915743-ddef-4d19-8675-57cfbcda9f72 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.039898 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1490/2960] 10.222.0.46 () {58 vars in 1122 bytes} [Thu Sep 1 07:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 07:58:39.043758 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.054089 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86ed5e-ffab-426a-a2a7-04e0b35ffb46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.058893 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.059520 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.059795 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.060065 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] PATH_INFO: `/v3/projects/0f519131ece94716bfa95ddf84bea562/users/06d5aba9086d4f29b6d84f1950339f1d/roles/55a4c413bf2645148a37872a21fe8eeb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.074810 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f519131ece94716bfa95ddf84bea562, user_id=06d5aba9086d4f29b6d84f1950339f1d, role_id=55a4c413bf2645148a37872a21fe8eeb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.077523 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.090116 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1471/2961] 10.222.0.46 () {60 vars in 1483 bytes} [Thu Sep 1 07:58:39 2022] PUT /identity/v3/projects/0f519131ece94716bfa95ddf84bea562/users/06d5aba9086d4f29b6d84f1950339f1d/roles/55a4c413bf2645148a37872a21fe8eeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:39.101003 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.101344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.101648 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.130697 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] MFA Rules not processed for user `06d5aba9086d4f29b6d84f1950339f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:39.132512 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:39.150108 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:39.198900 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1491/2962] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:39.210002 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.211716 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:39.267908 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510ec19-b13c-48e8-8df0-ebdbc90ac2e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.271785 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92e78fff3061449ba36aafbc1b3f7eaa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.272393 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.272668 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.272937 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.281530 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.282951 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f2999-5fff-4784-9ffe-359742aa7374 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.292184 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1472/2963] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:39.312032 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.372443 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b8954-5717-4cd3-a8b1-2acc89b987ee None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.377020 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.377669 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.377950 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.378224 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] PATH_INFO: `/v3/users/06d5aba9086d4f29b6d84f1950339f1d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.379344 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d5aba9086d4f29b6d84f1950339f1d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.381317 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.426125 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d5aba9086d4f29b6d84f1950339f1d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.426806 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d5aba9086d4f29b6d84f1950339f1d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.434218 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d5aba9086d4f29b6d84f1950339f1d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.434858 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d5aba9086d4f29b6d84f1950339f1d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.442415 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d5aba9086d4f29b6d84f1950339f1d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.460573 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1492/2964] 10.222.0.46 () {58 vars in 1224 bytes} [Thu Sep 1 07:58:39 2022] DELETE /identity/v3/users/06d5aba9086d4f29b6d84f1950339f1d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 07:58:39.465151 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.465440 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.465714 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.486585 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] MFA Rules not processed for user `82706f28e6f34da4b5745e5a95638706`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 07:58:39.487770 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:39.492207 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:39.548448 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1473/2965] 10.222.0.46 () {58 vars in 958 bytes} [Thu Sep 1 07:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 07:58:39.558654 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.560147 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:39.602719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:39.618293 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83589b3c-29cc-4007-af07-6e96ae8a276a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.621902 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '605f796f654b4b98aa29355fd4109aae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.622496 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.622771 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.623040 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.631601 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.633246 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.642360 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1493/2966] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 07:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 07:58:39.899013 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 07:58:39.900426 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 07:58:39.912657 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcd7f-b069-41d5-ba85-5ff1dbf2df88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 07:58:39.917858 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82706f28e6f34da4b5745e5a95638706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b911f14b05394fe89de7092f0075d554', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 07:58:39.918569 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 07:58:39.918838 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 07:58:39.919178 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] PATH_INFO: `/v3/projects/0f519131ece94716bfa95ddf84bea562` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 07:58:39.920360 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f519131ece94716bfa95ddf84bea562)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 07:58:39.922494 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 07:58:39.970365 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f519131ece94716bfa95ddf84bea562', 'request_id': 'req-e7b24802-91e2-42ee-9762-13dddad646e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.976402 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f519131ece94716bfa95ddf84bea562', 'request_id': 'req-e7b24802-91e2-42ee-9762-13dddad646e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.982063 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f519131ece94716bfa95ddf84bea562', 'request_id': 'req-e7b24802-91e2-42ee-9762-13dddad646e2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.994453 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] The token cache is being invalidate because project 0f519131ece94716bfa95ddf84bea562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 07:58:39.994740 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.995367 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e7b24802-91e2-42ee-9762-13dddad646e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 07:58:39.996863 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1474/2967] 10.222.0.46 () {58 vars in 1233 bytes} [Thu Sep 1 07:58:39 2022] DELETE /identity/v3/projects/0f519131ece94716bfa95ddf84bea562 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 07:58:49.638356 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=8 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 07:58:49.991258 dv-cinder-733062-35-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 08:02:54.097709 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 08:02:54.099719 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:02:54.212465 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199f35-007e-4a97-9715-797496351d0c service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 08:02:54.270652 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25fbbb7831ed429cb38cd7918e1923ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bce5454d9704d3b989a162e84789f3b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 08:02:54.271255 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 08:02:54.271532 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 08:02:54.271804 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 08:02:54.273547 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 08:02:54.284751 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 08:02:54.286977 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761e6fa-fe63-4ccd-af16-c9c4b22a4b77 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 08:02:54.296772 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1494/2968] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Sep 1 08:02:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 08:03:04.292412 dv-cinder-733062-35-iscsi devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=74 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}